Do The 10% Of Servicemembers Who Have Responded To The DADT Survey Represent A Statistically Significant Sampling?

Lez Get Real is reporting that of the “40,000 of the surveys that the Pentagon sent out to servicemembers have been completed since they were emailed out on 7 July to a mixture of active duty and reserve personnel.” “That is roughly ten percent of the 400,000 that they sent out, and Department of Defense officials need to hear from the rest regarding the repeal of Don’t Ask, Don’t Tell“:

The 200,000 active duty, and like number of Guard and Reserve, personnel have until 15 August to return the completed survey. The participants were selected randomly. Admiral Michael Mullen, Chairman of the Joint Chiefs of Staff, said that the Defense Department needs objective information and that the survey is the only way to get that. He emphasized that no one is drawing conclusions about the survey until it is finished. Admiral Mullen stated “To reach out at this point and try to predict either what they might say or what the results might say, I just think it’s too early with respect to that.” Unfortunately for Admiral Mullen, the survey may not bring in a large number of recipients due to a number of problems associated with online surveys. The number of turn ins have probably not been helped by the negative publicity that has surrounded the survey.

I wasn’t sure what to make of this news and unsure if the 40,000 surveys are a statistically significant amount from which the military can then extrapolate and reach some conclusions. Ryan McNeely tells me that it depends if the 40,000 represent a cross section of the 400,000 servicemembers or if those people were more likely to respond for some other unrelated reason. Without knowing that, it’s difficult to say if the results are representative of the force as a whole.

The point is that the servicemembers have until August 15th to fill out the survey and the Pentagon is encouraging them to do just that. The Defense Department disputes the 10% figure and says that the survey had to overcome several technical issues which has now resulted in an uptick of responses. They’re not putting out a specific number about what they expect the take-up to be or what they’ll consider “statistically significant,” but given that Secretary of Defense Robert Gates personally doubled the sample size, they seem intent on at least giving everyone the opportunity to weigh in on this, even if they won’t. And that’s of course precisely the problem with this kind of open-ended questionnaire: the only people who write back are those with the most extreme views and you end up knowing nothing about what the so-called “moderate middle” thinks. So who those 40,000 people are and how much weight the Pentagon lends to their opinions are all very significant.