Companion site for Design and Analysis of Experiments - 7th edition

http://bcs.wiley.com/he-bcs/Books?action=index&itemId=0470128666&bcsId=4460

Diet and Health

 

-          synopsis of parts of an article in forthcoming book, Evidence, Inference and Enquiry à In Praise of Randomization

-          Sausage a Day can increase bowel cancer risk.

-          2007 report, Food Nutrition, Physical Activity and the Prevention of Cancer, a Global Perspective.

-          Never been a randomized trial to test the carcinogenicity  of bacon

o   how strong is the evidence

o   turns out, surprisingly flimsy.

 

In Praise of Randomization

-          confusion of sequence and consequence

-          confusion of correlation and cause

-          Lady tasting Tea experiment   (tea test)

-          health studies

o   case control

§  least satisfactory  à retrospective

o   cohort

§  better à prospective

§  big effect à can give very good indication

o   Randomised controlled trials

§  best

·         only RCT can demonstrate causality

-          randomization

o   can remove in statistical sense à bias that might result from all sources you weren't aware of.

o   if you are aware of a bias, measure it.

o   guarantees freedom from bias in long run statistical sense.

-          case of hormone replacement therapy

o   Harvard Nurses Study à prospective chort

o   Women's Health Initiate Study à randomized double blind trial

 

Case of Processed Meat

 

-          recommendations only make sense insofar as various dietary factor cause cancer.

-          If association is not causal, changing diet won't help.

 

 

-          12 propective cohort studies showed increased risk for highest intake group compared to lowest

o   statistically sig only in three studies

o   tendency for relative risk to be above one, though not by much

-          Meta analysis possible on 5 studies

o   relative risk of 1.21

§  CI  1.04 to 1.42

-          consistency suggests real association, but this cannot be taken as evidence for causality

o   observational studies on HRT were just as consistent, but were wrong.

-          outcome from vast number of observations only just reaches p = 0.05 level of statistical sig

-          2 more criteria might help

o   good relationship between dose and response

o   plausible mechanism

-          no dose – response relationship apparent

-          study on vegetarians and cancer

-          pizza may simply represent a general and aspecific indicator of a favourable Mediterranean diet.

 

 

Is the Observed Association Ever Real

-          real associations likely to be exaggerated in size   à why

 

What do Randomized Studies Tell us

 

 

 

So Does Processed Meat Give You Cancer

 

-          only sound guide to causality is properly randomized trial

-          only exception when effects are really big

-          own inclination is to ignore any relative risk based on observational data if it less than about 2

 

Dangers of being too pre cautionary

-          ignore information that has sound basis

-          excessive medicalisation of everyday life

-          brings science into disrepute.

 

RESPONSES

observational epidemiology

Dr John Briffa

Michael Marmot

risks of second hand smoking

it doesn't matter if association is real if it isn't causal, because if it is not causal, it can't lead to useful action.

may be better to say we don't know

different types of sausages

whole idea of meta analysis is to look at all data

in observational studies, the statistical significance is not nearly as meaningful as the cofounders – those other factors that might explain the statistically significant effect observed.

how might high meat eaters differ from low meat eater  à good point.

the reason to do randomized trials is to render irrelevant all these possible cofounders

it always struck me that the very fact of having to do a meta analysis is pretty convincing evidence that the effect you are trying to nail down isn't real.

 

no amount of maths will extract information that isn't in the data

huge numbers of subjects helps in getting significant effects but with the risk you may end up simply detecting smaller and smaller associations that are ever more susceptible to misinterpretation because of cofounders.

information extend data à  Bayesian

 

jaynes book à logic

 

 

 

 

 

 

 

 

Engineering Statistics

http://itl.nist.gov/div898/handbook/index.htm

Engineering Statistics

http://itl.nist.gov/div898/handbook/index.htm

Experimental unit

Part 1 of this question asks what is the experimental unit.

The question states:

to test the effect of small proportions of coal in sand for manufacturing concrete, several batches were mixed under particularly identical conditions except for the variation in the percentage of coal. From each batch, several cylinders were made and tested for breaking strength.

The answer says the experimental unit is cylinder.

I suggest the experimental unit is batch, in that this is the unit to which the experimenter has applied the treatment.

The definition of experimental unit I am using comes from "Applied Linear Statistical Models" 5th ed by Kutner, Nachtsheim, Neter and Li, p652, and it notes

An experimental unit is the smallest unit of experimental material to which a treatment can be assigned; the experimental unit is thus determined by the method of randonmisation.

In this case, the batches would have had the proportion of coal randomly assigned to them, not the cylinders.

==============================================================================

"The smallest division of the experimental material to which we apply the treatments and on which we make observations on the variable under study, is termed as experimental unit". In this example the smallest unit is cylinder on which we make observation.

Experimental and Sampling Structures : Parallels Diverging and Meeting.

S Fienberg & J Tanur

International Statistical Review 1987 55 , 1, pp 75 – 96

 

-          prev  à randomization & random selection à fairness / objectivity / representativeness

o   novel departure in work of Fisher, Neyman / Tchuproc à introduction of chance mechanisms à to make available probability based methods of inference at analysis stage.

-          basic parallels between

o   design of randomized experiments

o   and

o   sampling studies

-          Example

o   two treatment randomization design for experiment

o   structure is identical to

o   selection of simple random sample

-          two modes of inference

o   design based inference à relies on probabilistic structure associated with design

o   model based inference , which introduces stochastic components as part of parametric structures.

 

Basic Parallels

 

-          randomization in experiments à probability / random sampling

-          both involving introduction of chance mechanisms

o   for assignment of treatments to units in experiments

o   choice of sample units in surveys

-          two treatment experiment

o   sample selection function function specifies which members of the universe are allocated to treatment 1, and which to treatment 2

o   in sampling situation, allocation to T1 corresponds to being selected for inclusion in sample

§  T2 corresponds to non selection.

-          purpose of randomization structure are different

o   experiment

§  compare

o   sampling

§  want to generalize from sample to rest of population

-          experiment – thru randomization, we hold everything constant.

o   thus, attribute any effects to treatment differences

-          sample

o   random selection and the fact that no treatment is applied à allows us to make generalization

-          both à randomization structure – used to provide meaningful estimate of variability.

 

 

-          split plot designs à analogous sampling technique, cluster sampling.

 

-          conceptualization of models for total survey error can take the component of variation due to interviewer as a random effect.

 

 

More Modern Parallels : Restricted Randomization

 

-          - blocking / randomization à introduced in agriucultural experiments à control for known heterogeneity in plots

-          this restricted randomization has applicability in sampling context

o   control for geographical spread of a sample

o   more generally, eliminate possibility of "bad samples"

-          because typical human population of interst in sampling is large and heterogenous, the simple device in restricted randomization in experimentation cannot be carried over directly.

 

 

Embedding

 

-          embedding experiments in sampling studies or sampling in experiments

-          while sampling to measure the outcome of an experiment was an intrinsic part of teachings of Fisher and of practice in agricultural experiments,

o   sampling yoked with experimental design is more rare

-          if experiments with surveys are to be of value, must apply the experimental principle of local control

-          interpenetrating networks of samples

o   design provided 5 independent estimates of economic conditions and as a consequence allowed for evaluation of the response variation associated with interviewers

-          large scale sampling à largest source of response error à associated with interviewer variability

-          random effects model

-          this suggests to those familiar with experimental principles of local control that a useful way to embed an experiment within a survey would be to use interviewers as a form of block.

 

Possible Causes and curses of Specialization

 

-          immediate analogue of treatments in sampling setting à samples

-          is it advantageous to have unequal probabilities of selection.

-          historically ,

o    many sample surveys were designed as enumerative studies

o   experiments – explore causal relationships

-          because analysis of non orthogonal experiments more difficult, many experiments designed to preserve orthogonality

-          surveys, rarely achieve orthogonality

-          simultaneity of inference

o   surveys

§  number of comparisons is enormous

§  fishing expeditions

 

 

Modeling and Inference

 

-          reporting of information from sample surveys – often cross classifications of frequencies  à descriptive or enumerative

-          models are essential for dealing with non-response and attrition

 

 

 

 

Module 2 : Topic 4 : Mortality Statistics and Standardization of Rates

 

-          Why study mortality statistics

o   vital statistics

-          Source of mortality data

o   death certificates

o   Australian Institute of Health and Welfare

-          Problems with death certificate

o   problem assigning one cause of death

§  death coded according to underlying cause of death

§  excludes info on immediate cause of death & inter mediate conditions

o   problem with accuracy and completeness of information provided.

§  medical opinions might differ.

§  some diseases under-reported à AIDS

§  some diseases over-reported à stroke

§  differences amongt medical practitioners in classification

§  international differences

§  use of International Classification of Disease codes

·         revised periodically

-          other sources of mortality data

o   autopsy

o   hospital records

o   occupational records

o   insurance / pension fund records

-          mortality studies

o   difference in mortality trends over time / between populations

§  may be artifactual  à result of errors in numerator or denominator

§  may be real

o   Artifactual

§  Errors in numerator

·         changes in recognition of disease

o   earlier detection of disease

·         changes in coding rules

·         changes in classification

§  Errors in denominator

·         errors in counting population

o   eg, under representation of young black males

·         errors in classifying by demographic characteristics

o   Real

§  changes in incidence of disease

·         genetic

·         environmental

o   EG, DECRAESED MORTALITY FROM ECTOPIC PREGNANCY

·         changes in age distribution of population

o   child birth – occurs in women of child bearing age, disappears as women reach menopause.

-          mortality statistics

o   rates

§  two denominators

·         mid year population

·         person years

o   total amount of time for which people were observed to be disease free during time of followup.

o   crude and specific rates

§  crude mortality rate (CMR) / Crude Death Rate (CDR)

·         all deaths during calendar year / mid year population

·         advantages

o   simple to understand & calculate

o   widely used

o   is a probability rate that a person belonging to population will die.

·         disadvantages

o   ignores age and sex distribution of population

o   specific mortality rates (SMR) / Specific Death Rate (SDR)

§  subdivision of data into homogenous subgroups

§  advantages

·         take into account age / sec composition of population

·         widely used

·         supplies essential components for construction of life table

§  disadvantages

·         not useful for comparing rates in different regions

·         in addition to age and sex distribution of population, social, occupational and topographical factors cause differential mortality

o   to avoid such spurious effects, standardized death rates are calculated.

o   cause specific mortality rate (CSMR)

§  common conditions à cancer, CHD

§  compare different populations

§  number of deaths specific cause / population

o   maternal mortality ratio

§  measure of risk of dying from puerperal causes – assoc with pregnancy

§  up to 42 days after termination / completion of pregnancy

·         puerperal deaths / live births

o   Infant mortality rate

§  deaths of infants under one year of age / number of live born infants

-          Age affects the rate

o   differences in age composition

o   crude rates à do not take into account age differences

o   versus adjusted / specific rates

-          standardized rates / adjusted rates

o   adjusted to control for effects of age

o   direct standardization

§  calculate age specific mortality rates for each age group in each population

§  select population whose age distribution is well defined to serve as standard or reference population

§  multiply number of people in each age group of reference population by age specific mortality rate in populations of interest.

§  sum total number of expected deaths

§  divide total number of expected deaths by total number of people in reference population

·         to yield summary age adjusted mortality rate.

o   Indirect Standardization

§  used when either age specific mortality rates not available

§  or they are statistically unstable

·         when population to be standardized is small.

§  SMR =  observed number of deaths / expected number of deaths

§  example – are death rates for male workers at certain company similar to death rate in population

·         choose reference population

o   make sure age specific mortality rates are known in reference population

·         calculate observed number of deaths in populations of interest

·         multiply age specific death rate from each age group in reference population by number of workers in corresponding age groups in company

·         sum the total number of expected deaths

·         divide total number of observed deaths in pop of interest by expected deaths

What Does IQ Really Measure?

http://news.sciencemag.org/sciencenow/2011/04/what-does-iq-really-measure.html?ref=hp

Role of test motivation in intelligence testing

1. Angela Lee Duckworth^a,¹,
2. Patrick D. Quinn^b,
3. Donald R. Lynam^c,
4. Rolf Loeber^d, and
5. Magda Stouthamer-Loeber^d

+ Author Affiliations

1. ^aDepartment of Psychology, University of Pennsylvania, Philadelphia, PA 19104;
2. ^bDepartment of Psychology, University of Texas at Austin, Austin, TX 78712;
3. ^cDepartment of Psychological Sciences, Purdue University, West Lafayette, IN 47907; and
4. ^dDepartment of Psychiatry, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213

1. Edited by Edward E. Smith, Columbia University, New York, NY, and approved March 25, 2011 (received for review December 14, 2010)

Abstract

Intelligence tests are widely assumed to measure maximal intellectual performance, and predictive associations between intelligence quotient (IQ) scores and later-life outcomes are typically interpreted as unbiased estimates of the effect of intellectual ability on academic, professional, and social life outcomes. The current investigation critically examines these assumptions and finds evidence against both. First, we examined whether motivation is less than maximal on intelligence tests administered in the context of low-stakes research situations. Specifically, we completed a meta-analysis of random-assignment experiments testing the effects of material incentives on intelligence-test performance on a collective 2,008 participants. Incentives increased IQ scores by an average of 0.64 SD, with larger effects for individuals with lower baseline IQ scores. Second, we tested whether individual differences in motivation during IQ testing can spuriously inflate the predictive validity of intelligence for life outcomes. Trained observers rated test motivation among 251 adolescent boys completing intelligence tests using a 15-min "thin-slice" video sample. IQ score predicted life outcomes, including academic performance in adolescence and criminal convictions, employment, and years of education in early adulthood. After adjusting for the influence of test motivation, however, the predictive validity of intelligence for life outcomes was significantly diminished, particularly for nonacademic outcomes. Collectively, our findings suggest that, under low-stakes research conditions, some individuals try harder than others, and, in this context, test motivation can act as a third-variable confound that inflates estimates of the predictive validity of intelligence for life outcomes.

Footnotes

• ↵¹To whom correspondence should be addressed. E-mail: duckworth@psych.upenn.edu.

• Author contributions: A.L.D., D.R.L., R.L., and M.S.-L. designed research; A.L.D., P.D.Q., D.R.L., R.L., and M.S.-L. performed research; A.L.D. and P.D.Q. analyzed data; and A.L.D. and P.D.Q. wrote the paper.

• The authors declare no conflict of interest.

• This article is a PNAS Direct Submission.

• This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1018601108/-/DCSupplemental.

Design and Analysis of Experiments - Chapter 4 Blocking / Latin Squares

 

Chapter 1 – Introduction To Designed Experiments

-          p 5 à see graphical representation of factorial experiment

-          randomization, replication, blocking

-          important distinction between replication and repeated measurements.

o   example -> four wafers are processed simultaneously in an oxidation furnace and then a measurement taken on oxide thickness of each wafer

§  this is repeated measurements

§  replication reflects sources of variation both between runs and (potentially) within runs.

-          Blocking

o   set of relatively homogenous experimental conditions

o   each level of nuisance factor is a block

-          recognition of and statement of problem

o   characterization or factor screening

o   optimization

o   confirmation

o   discovery

o   stability / robustness

-          design factors

o   design factors – factors selected for study

o   held constant factors

o   allowed to vary factors  à variations in experimental material à rely on randomization to balance out effects

-          nuisance factors

o   controllable  à blockable

o   uncontrollable  à analysis of covariance

o   noise

-          cause and effect diagrams – useful technique for organizing some of the information generated in pre-experimental planning.

-          Industrial era à development of response surface methodology

o   immediacy

o   sequentiality

-          robust parameter design

o   Taguchi

o   Wu

o   Kackar

 

Chapter 4: Experiments with Blocking Factors

 

-          nuisance factor

o   unknown and uncontrolled à randomization

-          known but uncontrolled

o   analysis of covariance

-          known and controllable

o   blocking

-          paired comparison problem

o   improve precision by making comparisons within matched pairs of experimental material

§  example – testing two tips

·         test each tip on same material

-          problem with completely random

o   experimental error will reflect both random error and variability between test beds

o   block – test each tip once on each test bed

o   randomized complete block

-          effects model

o   response = overall mean + treatment effect + block effect + error

o   if experiment were just completely random, variability for blocking would move into error

o   RCBD  à noise reducing technique

-          model adequacy checking

o   normality assumption

o   unequal error variance by treatment or block

o   block – treatment interaction

-          Some other aspects of randomized complete block design

o   additivity of randomized block model

o   where interactions are of interest à factorial design

o   fixed effects / random effects

 

Latin Square Design

-          is used to eliniate two nuisance sources of variability 

-          blocking in two directions

-          two restrictions on randomisation

 

 

 

Chapter 3: Measuring the Occurrence of Disease – Morbidity

 

-          Rates- how fast the disease is occurring in a population

-          Proportion – what fraction of population is affected

-          prevalence

o   numerator includes mix of people with different durations of disease

o   hence, do not have a measure of risk

o   measure of burdon of disease in community

-          problems with incidence and prevalence measures

o   problem with numerator

§  defining who has the disease

§  ascertaining which people should be included in numerator

·         how do we find cases

o   problem with denominator

§  selective undercounting of certain groups in pop – young men in ethnic minorities

§  different ways to classify by ethnic group

§  example

·         for rate to make sense, people represented by denominator  must have potential to enter group represented by numerator

o   hysterectomy /  uterine cancer rates

o   problems with hospital data

§  admissions policies

§  records not design for research

§  if we wish to calculate rates, have a problem defining denominator

·         hospitals do not have defined catchment area.

-          Relationship between incidence and prevalence

o   in a steady state situation, in which rates are not changing and in – migration equals out migration

§  prevalence = incidence * duration

o   example

§  extramarital births as percentage of total births in NZ

·         increased, but only because marital births decreased.

o   proportion is not a rate

o   birth can be viewed as an event

o   malformations à prevalence at birth

o   breast cancer incidence

§  distribution of cases by age

§  oldest age group has highest risk of breast cancern, but because this group is so small, only contribute small proportion of total number of breast cancern cases.

o   spot maps

§  clustering on spot map does not demonstrate higher incidence in area of cluster, for population most often clusters in that area.

§  many apparent clusters due by chance

 

Blocking

Module 2 Topic 1 Question 6

A researcher conducted an experiment to examine the efficiency of four types of fungal sprays (T1,T2,T3,T4) in controlling fungal rot on blueberries. Four adjacent rows of blueberries are available, each with 21 plants. Sprays can be applied to individual blueberry plants. The outcome / response variable is the proportion of blueberries with rot.

For the following design, specify experimental unit, blocking factor, and number of replications.

Each row is divided into 3 plots of 7 plants each. The sprays are randomly allocated to plots within each row.

Answer

Experimental unit = plot of 7 plants

Blocking factor is row

Number of replications is 3

=================================

I'm not sure if I totally understand blocking.

In Q6, there are 4 treatments.

There are also 4 adjacent rows, each row has 21 plants.

In part (b) of the question, each row is divided into 3 plots (of 7 plants each).

The treatments are randomly allocated to plots within each row.

If I'm reading it correctly, there are 4 treatments, but each block has only 3 spaces; therefore it's not possible to allocate all treatments to each block.

The answer says the blocking factor is row, but I understood each block should be a complete replication of the set of treatments.

??

===============================

Hi Graham,
The bit that confused me more about this was where the 3 replications came from, bu then I wrote this answer and figured it out :) It is late at night and I probably should have read this awhile ago! My reading of this is that by blocking each of 4 rows into 3 blocks you're left with 12 "units" to assign treatments to. If you start at row 1, randomly assign a treatment, move to row 2 randomly assign another treatment (without replacement) etc. Then start again at Row 1 with the same procedure, means you get the 4 treatments assigned 3 times each. My understanding is that blocking is a way of compensating for potential variation across your field that might influence your results but that you're not interested in. So by splitting your field up into smaller units this way and undertaking some sort of random assignment to blocks you might account for some of this impact. In this way you get your row blocking factor and your 3 replications because T1 is allocated 3 times across the field. I think this is it! I don't think it is necessary that each block is a complete replication of treatments? But I could be wrong. The agricultural experiments mess with me because it is so far outside of my field of expertise, I have to try and think how it relates to something I know and not easy!

Cheers,

Kate

=============================

Having worked in agricultural research, perhaps I should weigh in with my humble opinion :-)
Q6 (b) is a rather bad example.  In fact, by splitting each row into three plots, and then randomly assigning four treatments, until you have used up all 12 plots, you essentially get a fully randomised design, not a randomised blocks design.  By taking care to assign the four treatments of replication 1 to the first row plus one plot, then replication 2 to the remaining two plots of row 2 plus the next two of row 3, etc., you could conceivably get a randomised blocks design, but it's messy.

In randomised blocks, each block should be laid out in one part of the field.  A block doesn't have to be a uniform shape, but in practice, it normally is.  The main thing is that the plots making up a block should be grouped together in one area, to reduce the effects of fertility trends (soil heterogeneity).   Graham is correct:  In a (simple) randomised blocks design, the number of replications normally equals the number of blocks - at least, for the purposes of this course.  You can theoretically have as many treatments per replication (block) as you like, although you shouldn't - see below.

I say "normally", since the rule of thumb in field experiments is that a block should not exceed 16 experimental units (plots) in size, or the advantages of local control of soil heterogeneity are lost.  This means that for experiments with more that 16 treatments, which is quite possible in factorial designs, the use of ordinary randomised blocks designs runs into limitations.  With Latin squares, the situation is even worse.

For experiments with more that 16 treatments (or factor combinations), the researcher has to resort to more sophisticated experimental designs, in which several blocks per replication are employed.  These so-called incomplete blocks designs use deliberate confounding of certain treatment effects with block effects.  Higher-order interactions of factorial designs, which are generally of less interest to the researcher, are typically chosen for such confounding, and are dealt with in the analysis.  Here one also ventures into the area of so-called graeco-latin squares, and lattice designs, typically used in variety trials, where a large number of cultivars may be planted in a single trial.  Clearly, with varieties, you are not dealing with a factorial design, and so-called quasi-factors are employed in the design.   It is even possible to have single-replication designs, in which higher-order interactions are used to estimate error in the ANOVA.  Such experimental designs are clearly beyond the scope of this course, and I mention these just as a matter of interest.

Not surprisingly, much of the pioneering work in experimental design and analysis has come from agricultural research.  For those interested in this field (pardon the pun), the statistical package GENSTAT, developed at the Rothamsted agricultural research station in the UK, where Sir Ronald Fisher worked (with his associate Frank Yates) and developed ANOVA, is one of the few packages which makes a clear distinction between the blocking structure of an experiment and its treatment structure, and allows straightforward analysis of these fairly complex incomplete blocks designs. It was a nightmare in the days of hand calculations.   SAS also has origins in agricultural research.  The classic text to consult is Cochran and Cox's Experimental Designs.