Beyond simple penalized regression

Discussion in 'Scientific Statistics Math' started by meltwater, Dec 15, 2006.

  1. meltwater

    meltwater Guest

    We make predictive models for large datasets of 1e3...1e6 samples, with
    tens or sometimes over one hundred predictors. Usually the models are
    logistic regressions, but sometimes Poisson GLM's or linear regressions
    with t residuals. We use an in-house software and occasionally R. Our
    aim is almost entirely prediction -- rarely inference on the predictors
    influencing the outcome is needed.

    Our current approach is to use an isotropic gaussian prior for the
    regression coefficients, and then do MAP estimation. The accuracy for
    the prior is chosen by CV or DIC. The prior is there mainly to control
    model complexity in the presence of strong collinearity. We are not
    interested in model selection in the sense of minimizing the number of
    predictors.

    My question is: is there something better available for the kind of
    models we are doing? I'm mainly after an automatic way to choose the
    accuracy of the prior. Trying different values for the accuracy of the
    gaussian and doing CV on them takes so much CPU time that MCMC might
    actually be comparable.

    I have thought of trying hyperpriors for the accuracy and then
    sampling, or using an inverse Wishart prior for the covariance of the
    coefficients, instead of a fixed gaussian. Does anybody have pointers
    to literature along these lines, or any experience on related methods
    proposed in the literature?

    (I'm writing this anonymously partly out of convenience (to prevent
    spam), and partly because we are a company and would not like to share
    all the details we are doing with the competitors. Sorry for this.)
     
    meltwater, Dec 15, 2006
    #1
    1. Advertisements

  2. meltwater

    Reef Fish Guest

    That's a sufficiently bad thinking already. No model that's worth
    anything
    comes close to having that many predictors.
    You are a man with a bag of tools (actually just a shopping list of
    tolls)
    none of which you understand and none of which is suitable for your
    problem.
    Wise move. If your company name is known, you'll never get any
    business from anyone, just knowing the kind of mindless shufling
    of data into the Garbage-In Garbage-Out bins.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 15, 2006
    #2
    1. Advertisements

  3. meltwater

    Herman Rubin Guest

    If the t residuals have more than 3 degrees of freedom,
    it is unlikely to make much difference for linear regressions.
    Even more than 2 df might work reasonably well.
    What you need is not the accuracy of the prior, but the
    accuracy of the procedure. These are quite different,
    and it essentially takes a prior Bayes robustness approach
    to do this. It is possible to get reasonable procedures
    in many situations, including some in which the procedures
    may only look Bayes, or not even that.

    I can refer you to my paper with Sethuraman in Sankhya 1965,
    my paper on ridge regression in Bayesian Statistics 3, and
    the abstract and summary of the paper with Hui Xu and
    myself presented at the ASA meeting last summer, which
    will be his doctoral dissertation when expanded.

    Inverse Wishart priors do not have great properties,
    unless that aspect of the prior is of little importance.

    I suggest that further discussion be taken to email;
    it is getting too detailed to consider the other cases.
     
    Herman Rubin, Dec 15, 2006
    #3
  4. meltwater

    Reef Fish Guest

    Completely unsubstantiated ad hominem attack without a shread of
    evidence:

    RU> Reef Fish Bob speaks as someone with relatively little experience
    RU> at either data analysis or consulting.
    What has THAT have to do with the OP's questions about his
    Bayesian priors and superpriors without any understanding of
    what he was talking about?
    Utter SPECULATION completely unrelated to the OPs questions
    or my comments in the post, which FOLLOWED the opening
    sentence.
    As if you have anything useful to suggest about ANY statistical
    problem.

    You are such a Quack and unashamed malpractice ignoramus that
    I am running out of descriptive words to describe Richard Ulrich!
    What make you think the OP has anything to do with NetFlix? You
    did not address ANY of his questions

    OP> Our current approach is to use an isotropic gaussian prior for the

    OP> regression coefficients, and then do MAP estimation.

    Neither is even remotely Bayesian or statistical!

    OP> I'm mainly after an automatic way to choose the
    OP> accuracy of the prior.

    That is completely ANTI-Bayesian.

    OP> I have thought of trying hyperpriors for the accuracy and then
    OP> sampling, or using an inverse Wishart prior for the covariance of
    the
    OP> coefficients

    More meaningless and worthless name-dropping for something
    completely inappropriate for any statistical analysis with a Bayesian
    element in it.
    The OP is talking though his HAT, just as Richard Ulrich is ALWAYS
    talking through his HAT, and especially in his present post about
    NetFlix without addressing ANY of the items mentioned by the OP.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 17, 2006
    #4
  5. - Note Bob's claim, "NO model..." -
    Sorry, Bob. - Am I stepping on your private preserve?
    Only your own unsubstantiated ad-hominem attacks are okay?
    Okay, I'll substantiate a bit.
    I suppose I could claim that "as" can mean "like", and assert
    that my post was a refutation of Bob's claim. I give details
    of a model worth a million dollars to the *consultant*.

    But Reef Fish Bob has told us that he has not published
    analyses. He has regaled us with much of his career, and
    his "consulting" as a professor mainly for student problems.
    Moreover, any "relatively extensive" career in consulting
    would have rubbed off all those rough edges -- you know,
    immediately insulting so many people with questions is not
    the way to get *wide* experience, even if you see a lot of
    people.

    An experienced consultant, by my standards, has to be
    able to understand questions and take part in dialogs.

    Most everyone here does that better than Bob does.
    See? Bob does not understand the progress of the dialog.

    I refuted Bob. Then I went on to give details that other readers
    might find interesting, and to give a broader picture of when it is
    that hundreds of predictors might be useful.

    [snip]
    Bob does not understand the progress of a dialog.
     
    Richard Ulrich, Dec 18, 2006
    #5
  6. meltwater

    JS Guest

    I would not like to discuss the definitions of "statistics" and
    "bayesian", especially if the subjects are ridge regression and normal
    approximation to essentially gaussian posteriors -- this is what MAP
    with Fisher information are here.
    There is a whole field of research trying to find flexible, widely
    applicable models with few adjustable parameters. Google for
    "nonparametric bayesian" or "probabilistic machine learning". The
    methods are bayesian in the following sense: (1) the probabilistic
    machinery and bayesian inference are used; (2) models are generative;
    (3) there is a fixed prior, but it is on a very general level, ideally
    domain-free. I don't know what makes the approach anti-bayesian - even
    in traditional statistics (with less automation and more
    domain-specific knowledge) hyperpriors are used that do not strictly
    encode anyone's prior information. Quite the contrary, they are often
    trying to be "non-informative".

    I'm after a similar kind of solution for our problems. Or actually we
    already _have_ that kind of solution that is working pretty well. Our
    current methods are just based on innovations two decades old, and
    given that things have advanced pretty well on related fields, I
    wondered what is the state of the art with regression. I'm aware of
    SVM's , "relevance vector machines", sparse regression with Laplacian
    priors, and the interpretation of Laplacian priors as a hierarchy of
    gaussians and Jeffrey's. All this, however, is a bit beside the point
    for us, for we have too much data for the kernel approach (SVM, RVM),
    and no need for sparsicity.

    Of course, models or model complexity can never be chosen blindly and
    automatically, at least not if one is after more than mediocre results.
    ThereƤ's even theorems on that. But some model families and procedures
    are more widely applicable than others, _including_ the complexity
    dimension. You can say that they fit better with our prior knowledge,
    if that helps.

    I thank Herman Rubin for the pointers to his papers on ridge
    regression.

    Look at the Netflix competition. The data set is so huge that the best
    methods currently available for collaborative filtering (such as URP
    and its derivatives) may not be applicable. But those methods are
    specifically tailored for large data sets. They approximate the
    posterior of an additive mixture model with variational methods. The
    models have thousands of parameters, and a comparable number of latent
    variables. The Netflix dataset cannot be analyzed with even that amount
    of bayesian and technical pureness. (I'm not affiliated with Netflix,
    neither as an employee or as a participant in the competition.)

    (Dear Bob Ling: please, do not feel obliged to reply if you are
    emotionally or intellectually incapable for discussing this. I wrote
    this mainly for other potential readers.)
     
    JS, Dec 18, 2006
    #6
  7. meltwater

    Reef Fish Guest

    You have just given the REASONS for my statement!

    MAP has been thoroughly discussed in this group by Illywhacker, a
    physicist nonBayesian, to think that it's Bayesian; and Ridge
    Regression
    is certainly nonBayesian, and has been out of favor since the 1980s
    for the inappropriate reasons (wrong sign of regression coefficient
    by those who don't know what regression signs mean) for which the
    procedure was used.

    For the simple reason that a "prior" is supposed to represent the
    user's OPINION about the parameters, not something automatically
    chosen from DATA.
    There is also an entire subculture of sociologist, economists, and
    other NON-statisticians who practice statistical Quackery, which
    is well-known by statisticians to be quackery, and many of them
    are even in this group!
    The last line is why its anti-Bayesian. That's the LAST thing a TRUE
    Bayesian want his prior to be, "non-informative". You should read
    some books and articles by Bayesians on Bayesian statistics, rather
    than the malpractice examples in field OUTSIDE of statistics proper.

    You are NOT a Bayesian. I am one. I learned my Bayesian statistics
    from L.J. (Jimmie) Savage, Harry Roberts, Schleifer, and a host of
    other
    Bayesians in the world of Statistics.
    Here's a paper you should perhaps read, to find out WHY apart from
    the fact that Ridge Regression is chosen for the wrong reasons, it
    is also a poor substitute for the criterion of minimizing the MSE in
    a regression. It's a paper by my former doctoral student (based on
    his dissertation on the subject) and myself in the Journal of
    Statistical Computations and Simulations (1979):

    http://jscs.statjournals.net/ARTICLES/v09n03.html

    Ridge Regression has virtually disappeared in JASA and other
    reputable statistical journals of Statistics.
    NetFlix is NOT statistics. It's dataset is for exhaustive search and
    ID based on a very special data that is only used BY NetFlix. It has
    no use, nor is it usable by, or useful to, the field of statistics.
    I do feel obliged to reply for the reason that YOU are emotionally
    and intellectually immature and completely uninformed in those
    Statistical topics related to your problem that you are unable to
    take the VALID criticisms of your blind data dredging and misuse
    of Bayesian statistics and Bayesian ideas as if you're doing
    something valid in statistics.

    I am writing this, with much more detailed REASONS given, why
    any properly trained APPLIED statistician who is familiar with
    Bayesian statistics and Bayesian methods would have
    understood the one line on what I had to explain to YOU in
    many paragraphs.

    Your own ignorance in statistics and your lack of ability to face
    valid statistical criticisms directed toward your ABUSE and
    MISUSE of statistical and quasi-statistical methods showed
    clearly and unequivocally in your posts in this thread.

    That is one reason you found sympathy in the biggest Quack
    and statistical malpractice author in the sci.stat. groups,
    Richard Ulrich.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #7
  8. meltwater

    JS Guest

    I don't know what to make out of this, except that you are unfamiliar
    with the equivalence of ridge regression and exchangeable gaussian
    prior on the regression coefficients.
    I could also have said "I would like to express my vague prior
    knowledge I'm willing to put into this as a hyperprior for the gaussian
    rather than as a fixed gaussian prior", if it makes you feel better.

    This is a context where the prior information wanted on the model is on
    a very general level, like "I find large coefficients on collinear
    predictors unlikely".
    I don't know who is a True Bayesian, and I don't care. But by rejecting
    non-informative hyperpriors, you are practically redefining bayesianism
    as applied in _traditional_ statistics today. To understand this, see
    some of the modern reviews (books), such as Bernardo and Smith, Gelman
    et al., Gilks et al.

    Even in the presence of good application-specific knowledge, there are
    many perfectly valid reasons for using non-informative priors, part of
    them technical and part procedural. You should know this if you have
    ever used even a simple linear model.

    One could also ask: why would you use a linear model if you are the
    kind of True Bayesian you claim to be? Surely a linear model never
    presents your prior knowledge accurately. Linear models are an
    artificial subset of a much larger model family - by restricting
    yourself to linearity without any deviations, you are essentially using
    a highly artificial prior on that larger model family, and that surely
    does not accurately present your prior knowledge. Are you excusing on
    the basis of technical reasons, laziness or what? Why don't you put
    your real prior information into your model?

    (Well, I guess you have rejected linear models long ago.)
    Fine with me. I have a problem and I want to solve it. If I have a
    religion, it is nontechnical, or at least not related to formalisms for
    handling uncertainty.
    Thanks - although a lot has happened since late 70's.
    One could also say that it lives in every hierarchical model, but
    please do not start to argument about that.
    Again, I don't want to discuss semantics on such a general level here
    and right now.

    There are ways to handle such datasets that use statistical and
    bayesian techniques and formalism. They exist, and are rigorous, and
    effective, no matter how you or me want to call them.

    If someone asks a technical question here on a context that is not
    familiar to you, it would help this group a lot if you would shut up
    instead of attacking the questioneer. If you can't keep quiet, at least
    reply with substance, not with an attack. This used to be a relatively
    civilized group with high signal-to-noise ratio, but now the situation
    seems surprisingly poor - so poor that a moderated group is starting to
    look attracting. You could, personally, help a lot by practicing a bit
    of moderation on yourself.
     
    JS, Dec 18, 2006
    #8
  9. meltwater

    Reef Fish Guest

    Your inference is correct. Both of these are useless to me. Ridge
    Regression can also be given some pseudo-Bayesian interpretations,
    but that doesn't make it any more useful or legitimate as a statistical
    method.
    No. It make you and other nonBayesians feel like they should demand
    respectibility by throwing the term Bayesian around because it seems
    fashionable in some circles. You seemed to be completely unaccustomed
    to the fact that priors are NOT to be taken likely by a true Bayesian.
    All your terms are just convenient concepts for the lazy-non-thinking
    non-Bayesians who think they are using Bayesian methods.
    Why pretend to be a Bayesian? Why not just use legitimate nonBayesian
    regression methods?

    The fact that YOU can make the statement

    JS> "I find large coefficients on collinear predictors unlikely".

    NOT knowing what each and everyone of all the predictors used in the
    regression is a 100% unmistakable indication that you DON'T KNOW
    how to interpret the coefficients in a regression. The fact that you
    think "collinear predictors" unlikely when you have even dozens,
    let along hundreds (in your case) of predictors is a sure sign that
    it's your unsubstantiated wishful thinking, completely void of any
    theoretical or empirical basis.
    That's rather obvious. Your statement is merely a redundant
    expression
    of your ignorance in the subject.
    There does NOT EXIST any true Bayesian in the world in APPLIED
    statistics who has to deal with more than 2 or 3 parameters because
    NONE of them know how to solicite and represent their priors. They
    only use the pseudo-Bayesian ideas to fool themselves and others
    like yourself.

    Just don't PRETENT to be a Bayesian, if your forte is the use of
    noninformative priors. Just try to learn the theory and method of
    tranditional nonBaysian and Data Analytic techniques that, as
    imperfect as they are, are infinitely more understandable and
    respectable than being a Bayesian Quack -- which takes the WORST
    of both worlds -- Bayesian and non-Bayesian.

    That is why I don't PRETEND to be a Bayesian in my analysis of
    linear models because it has its valid and useful techniques
    WITHOUT the use of prior information which are NOT expressible
    to reflect the true prior beliefs.
    You step into this neighborhood completely UNARMED in your
    knowledge about any form of statistics, and completely unaware of
    the thosands of articles I have posted since early 2005 in this
    group about the PROPER use of linear models and model-building
    methods -- that are obviously far beyond the level of your
    statistical education.
    That's the characteristic of a Quack. Untrained in medical sciences,
    he thinks he can cure all ills. A statistical Quack is of the same
    ilk
    I see your kind everyday, in THIS forum.
    You typed before you read the sentence below.
    Like Factor Analysis, Ridge Regression has lived its brief life in
    the statistical literature (while there were still some hope that they
    have something to deliver). Since their 15 months (or years) of
    fame, they are not completely abandoned by anyone worth his
    salt in the field of Statistics. I have heard (from other editors)
    that there was a moratorium (though unofficial) on the publication
    of ALL Factor Analysis papers in JASA. That was in the decade
    when I was an Associate Editor of JASA, in the 1970s to early
    1980s. That moratorim seems to be in effect ever since, and so
    is the application of Ridge Regression for anything!
    You are merely repeating your buzz words in which you have no
    knowledge and understanding that they are NOT valid nor justifiable
    statistical methods.
    I have, in my detailed response to your uninformed and uneducated
    allegations, GIVEN you and everyone else the context of those
    areas that they are VERY familiar to me -- which is why I could
    reject them as Quackery!
    I replied with substance that you didn't even RECOGNIZE. That is
    how deficient you are, in those areas in which you are just throwing
    around a few words you read from other Quacks.
    Look at the history of Richard Ulrich's Quackery from 1995 through 2004
    -- whose ERRORS and malpractices were not corrected or challenged
    by ANYONE in this group.

    It's high signal-to-noise alright, if you consider Quackery-Signal as
    signal.

    There has been plenty of Statistical Signals (hundreds of them on
    my correction of Richard Ulrich's ERRORS alone -- they are ALL in
    the sci.stat.math). If you know how to use the Google archives,
    you can look up ANY statistical term in the advanced search of
    group.google.com, together with author "Reef Fish", you WILL find
    plenty of statistical signals, among also plenty of statistical NOISE,
    by Richard Ulrich, Anon Bob O'Hara, Luis A. Afonso, Greg Heath,
    and another handful of error-making NOISE-makers.
    Go peddle your statistical Quackery in some other groups. YOu'll
    find company in the groups Illywacker peddles his statistics by
    physicists who call themself Bayesians. You can also find 100%
    of your signal to no noise in the new group scistatmath formed
    by beliavsky because he didn't like the NOISE in this group.

    The scistatmath group is where you belong.

    Well, I just noticed your post of your OP in THIS thread there,
    on Dec 15 -- the only post in that group since beliavsky's post
    of "distribution of things" (now THAT's garbage is I ever seen
    one in a statistical group).

    Why are you back HERE making YOUR noise? Aren't you
    perfectly happy with all the help you got in that group with
    absolutely no signal and no noise?

    You've had your opportunity to EXPOSE your own Quackery
    while making your NOISE.

    You should continue your subject in some physics/software/
    groups that specialize in the malpratice of statistics -- you'll
    find lots of signal of the kind you like to hear, and will be very
    happy there.

    Your continued presence in this thread in this group merely
    confirmed what I said in my reply to you:

    RF> I do feel obliged to reply for the reason that YOU are emotionally
    RF> and intellectually immature and completely uninformed in those
    RF> Statistical topics related to your problem that you are unable to
    RF> take the VALID criticisms of your blind data dredging and misuse
    RF> of Bayesian statistics and Bayesian ideas as if you're doing
    RF> something valid in statistics.

    Your free consulting session is OVER. I think Richard Ulrich is
    out there eagerly waiting for you to contact him to be your paid
    consultant, because he had all kinds of advice for you on NetFlix
    and he must be your IDOL in this group because he had been
    the most profilic, made most errors, and made most NOISE in
    his years in sci.stat.math.

    May you two find true happiness embracing each other and
    blissfully share your Quackeries.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #9
  10. meltwater

    Bob O'Hara Guest

    Oh. I was thinking of sitting down and reading this book over Christmas:
    O' Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. E., Garthwaite, P.
    H., Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006). Uncertain
    Judgements: Eliciting Expert Probabilities. John Wiley and Sons.

    Can you explain to me what's so bad about it, please. I'm assuming you
    know Tony O'Hagan's work, otherwise you wouldn't make the comment you did.

    Bob
     
    Bob O'Hara, Dec 18, 2006
    #10
  11. meltwater

    Reef Fish Guest

    Bob O'Hara wrote:

    goaded by the foot in his mouth that had been kicking him to open:
    You never READ, even the most obvious -- even the paragraph you
    CITED -- that was the FULL explanation needed.

    You are keeping your batting average to be 100% WRONG.

    Tony O'Hagan? Never even HEARD of him until you mentioned now.

    http://www.itee.uq.edu.au/~tohagan/?print=1
    http://ctcd.group.shef.ac.uk/ohagan.html

    I might have guessed the australian computer jock, except for the
    8 coauthors who are affiliated with the U of Sheffield.

    I CERTAINLY would make my comment without reading ANYTHING
    any of them wrote about Bayesian statistics.

    Show me ONE single example of their elicitation and representation
    of prior in 4 dimensions, or anyone else in the world, and you'll
    have a POINT. Not until then, you're just wasting your time
    reading garbage -- which is the foundation of all your Quackery.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #11
  12. meltwater

    JS Guest

    (Sorry for e-mailing you, I have problems with the software.)

    Here's a couple of reasons why someone would like to put incomplete
    prior knowledge into a model:

    (1) He/she is actually a machine with limited interaction with the
    domain experts, and therefore in a sense equipped with poor prior
    knowledge. This situation applies to really autonomous machines as well
    as statistical modelling applied in a context where the model cannot
    interact with domain experts. The reason for the latter could be
    economical, for example.

    (2) The expert wants the model to be objective.

    (3) Related to the above, bayesian framework can be used _internally_
    within the model, for example in a hierarchical model, where it allows
    propagation of uncertain knowledge.

    (4) Even poorly expressed prior information may be better than no prior
    information at all.

    (5) Related to the above, by choosing a model family even a frequentist
    puts prior information into the model. The distinction between a prior
    and model is in principle artificial, and getting more and more
    artificial in practice (with nonparametric models).
    Of course. And I do not want to interpret them, because I'm after
    predictions, not after understanding the details of each and every
    model.
    I have lots of empirical support for the success of my approach. BTW, I
    didn't say I find collinear predictors unlikely. I said that I find
    collinear predictors with large (opposite-sign) coefficients unlikely -
    not impossible.
    Or: I found most of the models found by traditional regression highly
    unlikely (a priori), because the coefficients jump all around on the
    proverbial walls.
    You mean you implicitly use noninformative, improper priors?
    Or worse yet, step-wise variable selection?
    Tell me, what do the current editors of JASA think of hierarhical
    bayesian models? It's easy to guess: the editors are quacks. When did
    the change happen?
    This pretty much summarizes your position at the marginal of
    contemporary statistics. It also helps to understand your frustration
    and hate. You are bayesian by heart, but forced to use frequentist
    practices because of your pureness. That's a really tight place to be.

    Peace,
     
    JS, Dec 18, 2006
    #12
  13. meltwater

    JS Guest

    How is it, by the way, that by simple googling (not searching JASA
    abstracts) I find several papers on factor analysis and associated
    latent variable models from recent years (since 2000)?

    Also penalized regression (ridge regression) seems to applied in a lot
    of papers, at least in the context of flexible models like splines.

    Have you actually read JASA since 70's?
     
    JS, Dec 18, 2006
    #13
  14. meltwater

    Reef Fish Guest

    That's one of my many (inconsequential) typos in every post.
    Email? I am reading this public post in groups.google.com now.
    Ignorance is no excuse for MISUSE.
    That's true. But NONE of yours reflect any Bayesian prior.
    Then how DO you know about how large or small the coefficient
    should be? Or lack of multicollinearity?

    I had already given you too much additional free consulting.
    Go get Richard Ulrich to help you. He has solution to every
    contemporary problem in statistics, sociology, epidemiology,
    and of course YOUR problem.

    You characterization is not far from correct, except for the use of
    "hate". Yes, it is not easy to be CORRECT when the majority of
    THIS crowd are in the wrong and don't recognize it. It's the same
    situation to have an IQ of well over 150 and try to talk some sense
    into folks of "average" intelligence who happen to be obtuse and
    resistence to knowledge.

    Just put yourself as an average person (I am assuming you are;
    and if you think you are a bit above, it's okay with me too) and try
    to teach some statistics or common sense to IDIOTS (technical
    definition of IQ less than 50) and you'll see my "tight place".

    Being one of the crowd has never been, or ever will, be my
    aspiration, no matter what popularity award there dangles.
    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #14
  15. meltwater

    Bob O'Hara Guest

    In fairness to Reef Fish, I think he retired before the modern Bayesian
    wave gathered pace, so he wasn't following developments, and is unaware
    of recent developments (such as how hierarchical models are used).

    Bob
     
    Bob O'Hara, Dec 18, 2006
    #15
  16. meltwater

    Reef Fish Guest

    If you want to have anything of substance to contribute, why don't you
    find ONE single Bayesian prior of 4 dimensions (or higher) that was
    realistically elicited and assessed -- you have unlimited time span
    to choose from -- 1900 to 2007 and beyond.

    If you can't find any, then you should SHUT UP.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #16
  17. meltwater

    Bob O'Hara Guest

    I've got some BUGS code from a colleague with 5 dimensions. Mary Kynn
    presented an example at IceBUGS with 37 variables:
    <http://www.math.helsinki.fi/openbugs/IceBUGS/Presentations/KynnIceBUGS.pdf>

    What this has to do with hierarchical models (the example I used of
    developments which I suspect Reef Fish to have missed) is another matter.

    Bob
     
    Bob O'Hara, Dec 18, 2006
    #17
  18. meltwater

    Reef Fish Guest

    I didn't ask for any PROGRAM or CODE. I asked for an example of
    the realistic ELICiTATION and REPRESENTATION of a person's
    realistic prior.

    WHat you have shown is your own TOTAL ignorance in the subject
    of elicitation of prior distributions.
    YOur comments are entirely irrelevant.

    All you know are how people in your area ABUSE the use or prior
    distribution and ABUSE the use of programs. A typical example of
    Garbage In - Garbage Out.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #18
  19. meltwater

    Reef Fish Guest

    What you have here is a TYPICAL example of a MINDLESS, useless,
    meaningless pseudo-application of the process of elicitation of a
    Bayesian's prior distribution.

    TWO immediate errors occur even before the pseudo-elicitation phase
    started:

    1. Assume the PREDICTORS are statistically independent!
    That's one of the BLUNDERS that had been discussed at length
    in the sci.stat.math group. The independent variables in a
    regression are LINEARLY independent but not stochastically
    independent!

    2. The ASSUMPTION that a normal distribution is the form of the
    prior for each coefficient.

    That disqualified the author so badly that there is no need to
    read anything else about the quackery.

    Garbage-IN, Garbage-OUT.

    I predicted that with 100% accuracy before I read anything published
    or unpublished by these Quacks.

    Show me a REALISTIC elicitation of a PRIOR -- you CANNOT put
    restrictions on what form that prior should take, or to what extent
    the parameters (variables) in the prior are statistically independent
    or to what extent they are dependent -- THOSE should be determined
    by the elicitation procedure.

    For a ONE Variable realistic elicitation, it may take hundreds of
    questions.

    For TWO variable prior, it may take tens of thousands of questions
    to determine the BIVARIATE distribution shape.

    You are just doing the fool's dance thinking that some BUGS code
    or BUGS BUNNY code can do that for you.

    As I had said before, you are NOT a Bayesian and you never
    understood anything in Bayesian Statistics other than the
    Quackery you encounter in your group of non-statistician
    programmers.

    -- Reef Fish Bob.
     
    Reef Fish, Dec 18, 2006
    #19
  20. meltwater

    Bob O'Hara Guest

    Which is what I gave you. Read the slides of the talk.


    Bob
     
    Bob O'Hara, Dec 19, 2006
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.