Analyzing Subpopulations in Complex Survey Data

Using Stata for Subpopulation Analysis of Complex Sample Survey Data
Brady T. West PhD Student Michigan Program in Survey Methodology
July 30, 2009
2009 Stata Conference
Presentation Outline
1. 2. 3. 4. Introduction: Subclass Analysis Issues Kishs Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons 5. Examples using NHANES and NHAMCS Data 6. Suggestions for Practice 7. Directions for Future Research
2009 Stata Conference: Subpop Analysis of Survey Data 2
Subclass Analysis Issues

Analysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey data
Subclass Analysis Issues, contd

Software procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference
Kishs Taxonomy of Subclasses

Design Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., AfricanAmericans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for Research
Design Domains
X = Sample Element in Subclass Stratum PSU 1 PSU 2
1
2 3 4 5
XXXXXXX XXXX XXXXXXX XXX
XXXXXXX XX XXXXXXX XXXXX
2009 Stata Conference: Subpop Analysis of Survey Data
Cross-Classes
Stratum 1 2 PSU 1 PSU 2 XXXXXXX XXXXX XXXXX XXXX XXXXXXX
3
4 5
XXXXXXX XXXXXXX XXXX XX XXXXXX XXXXX

XXXXXXX XXXXXXX XXX XXXXX
Mixed Classes
Stratum 1 2 PSU 1 PSU 2 XXXXXXX XXXXXXX XXXXXXX XXXXXX X
3
4 5
XXXXXXX XXXXXXX XXXXXX XXX XX

XXXXXXX XXXXXXX XXXXXXX XXXXX
Applying Kishs Taxonomy

The type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains)
The Unconditional Approach

Appropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass!

Rationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!

Further, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures
12

If cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedom
The Unconditional Approach: General Stata Code

svy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations
* Exercise: repeat 10 times really fast
The Conditional Approach

Appropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom
The Conditional Approach

Restrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-tosample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by design
The Conditional Approach: General Stata Code

svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)
17
Variance Estimation Methods

All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)
Ad-hoc Fixes for Singleton Clusters in Stata 10.1

Stata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis:
1. Report Missing Standard Errors (not really a fix) 2. Treat Units as Certainty Units, which contribute nothing to the standard error 3. Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU 4. Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum mean
Example: The NHANES Data

We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 (oldie but a goodie) Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population
Example NHANES Analysis

Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more details
Conditional Approach: Stata Code for NHANES Analysis

svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1
Conditional Approach: Results

Method Missing SE Est. Mean 144.09 TSL SE . Design DF 50-29 = 21
Centered
Certainty Scaled
144.09
144.09 144.09
1.66
1.62 1.90
50-29 = 21
50-29 = 21 50-29 = 21
23
Conditional Approach?
This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!
Unconditional Approach: Stata Code for NHANES Analysis

svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst
25
Unconditional Approach: Results

Method Missing SE Centered Certainty Est. Mean 144.09 144.09 144.09 TSL SE 1.66 1.66 1.66 Des. DF* 58-29 = 29 58-29 = 29 58-29 = 29
Scaled
144.09
1.66
58-29 = 29
* Note: Stata dropped three strata with no sample units in the subpopulation.
Unconditional Approach?
This approach would be the appropriate choice for a cross-class such as AfricanAmericans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable
Example: The NHAMCS Data

Analysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details
Stata Code for NHAMCS Analyses

svyset cpsum [pweight = patwt], strata(cstratm) singleunit() generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional
NHAMCS Analysis Results

Method
Missing SE
Est. % 4.82
TSL SE 1.576
Design DF 106
Centered
Certainty Scaled Unconditional
4.82
4.82 4.82 4.82
1.576
1.576 1.576 1.590
106
106 106 286
30
NHAMCS Analysis Implications

No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-tosample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences
31
Suggestions for Practice

Consider Kishs Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing crossclasses Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)
Directions for Future Research

More appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results
References
Kish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 112. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings.
Questions / Thank You!

For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edu
35

Analyzing Subpopulations in Complex Survey Data

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Analyzing Subpopulations in Complex Survey Data

Caricato da

Copyright:

Formati disponibili

Using Stata for Subpopulation Analysis of Complex Sample Survey Data

Brady T. West PhD Student Michigan Program in Survey Methodology

July 30, 2009

2009 Stata Conference

Subclass Analysis Issues

Subclass Analysis Issues, contd

Kishs Taxonomy of Subclasses

XXXXXXX XXXX XXXXXXX XXX

XXXXXXX XX XXXXXXX XXXXX

2009 Stata Conference: Subpop Analysis of Survey Data

XXXXXXX XXXXXXX XXXX XX XXXXXX XXXXX

XXXXXXX XXXXXXX XXXXXX XXX XX

Applying Kishs Taxonomy

The Unconditional Approach

The Unconditional Approach

The Unconditional Approach

2009 Stata Conference: Subpop Analysis of Survey Data

The Unconditional Approach

The Unconditional Approach: General Stata Code

The Conditional Approach

The Conditional Approach

The Conditional Approach: General Stata Code

2009 Stata Conference: Subpop Analysis of Survey Data

Variance Estimation Methods

Ad-hoc Fixes for Singleton Clusters in Stata 10.1

Example: The NHANES Data

Example NHANES Analysis

Conditional Approach: Stata Code for NHANES Analysis

Conditional Approach: Results

2009 Stata Conference: Subpop Analysis of Survey Data

Unconditional Approach: Stata Code for NHANES Analysis

2009 Stata Conference: Subpop Analysis of Survey Data

Unconditional Approach: Results

Example: The NHAMCS Data

Stata Code for NHAMCS Analyses

NHAMCS Analysis Results

2009 Stata Conference: Subpop Analysis of Survey Data

NHAMCS Analysis Implications

2009 Stata Conference: Subpop Analysis of Survey Data

Suggestions for Practice

Directions for Future Research

Questions / Thank You!

2009 Stata Conference: Subpop Analysis of Survey Data

Potrebbero piacerti anche