Survey Research Support

Considerations

Depending upon the population and the topic of your survey, we suggest considering the following issues:

PERMISSIONS

Before administering a survey to a population, and therefore emailing a large segment of that population, it is important to notify and get permission from the senior officer responsible for that population.

• For Undergraduate students, that would be the Dean for Student Life, the Vice Chancellor, or the Chancellor

• For Graduate students, the Vice Chancellor or the Chancellor

• For Post-Docs, the Vice President for Research

• For Faculty, the Chair of the Faculty or the Provost

• For Staff, the Vice President for Human Resources, the Executive Vice President and Treasurer, or the Vice President for Institute Affairs

ANONYMITY AND CONFIDENTIALITY

There are different types of surveys: polls that ask for respondents to give input for the planning of an event, a data collection asking respondents to provide information to update their records, or engagement surveys that seek to gauge respondents’ attitudes toward a set of topics. Depending upon the type of survey and the subject matter, it is important to determine whether the survey should be anonymous, confidential, or neither.

• Anonymous: It is difficult to guarantee anonymity. Many survey platforms log IP addresses, and many individuals can be uniquely identified by a combination of demographic data points that are commonly asked on surveys. . For example, if the combination of age, gender, ethnicity and major would allow the surveyor to know the identity of a specific individual, then the survey does not meet the standard of being anonymous. A downside to an anonymous survey is that it is difficult to restrict the responses to be one per individual. However, for sensitive subject matters, anonymity may yield more honest responses.

• Confidential: In MIT Institutional Research, we most often conduct surveys that are confidential but not anonymous. We tend to authenticate our surveys (i.e. we know who is answering using unique links) and maintain some basic identifying information along with the responses. This allows us to analyze responses along a variety of demographic factors, but also gives us the responsibility of guarding sensitive information.

  If the administrator of a survey promises or implies that the responses will be treated as confidential that means that the results are confidential to the research team; i.e., the results should never be reported at a level of granularity that would allow someone outside the research team to infer the identity of an individual. In no cases should individual survey responses be shared with a supervisor, an advisor, or anyone who is not part of the research team. Whenever possible identifiers should be separated from the dataset.

  There will be cases where the research team includes people in authority or people who have a relationship with individuals being surveyed, such as the personnel administrator for a department or a housemaster. In these cases, it must be made very clear to those invited to take the survey who will have access to the individual responses

  Occasionally, existing datasets will be requested for uses beyond the original intent of a survey. For example, a senior officer may want to have additional analysis done on a particular topic or an academic researcher may want to use data collected in a survey for their research. In these cases, the analyst will be bound by the same confidentiality requirements as the original team.

INCENTIVES

Depending upon the population and needs of your survey, it may be desirable to offer incentives to survey respondents. Two guidelines to consider are that the incentives must be positive (i.e. no negative consequences for non-respondents) and should be relevant to the topic of the survey (e.g., if you are conducting a transportation survey with the purpose of encouraging public transit, free parking passes would not be an appropriate incentive).

RESEARCH WITH HUMAN SUBJECTS

If you or anyone working with the data will be analyzing data that identifies individual respondents, either by name, email address, MIT ID number, or other unique personal information, then those staff members working with the data will need to complete COUHES (Council on the Use of Human Experimental Subjects) training prior to handing sensitive data. This training can be done from any computer with MIT certificates enabled from the URL: http://couhes.mit.edu/ 

If the data are to be gathered and used as part of an academic research project, the project will need to be formally cleared with COUHES to ensure that it meets their requirements. If the data are to be used for administrative purposes, it may be exempt from a formal review by COUHES, but the survey must be voluntary and the results kept confidential. Please consult with IR if you have any questions about COUHES requirements.

INCLUSIVE LANGUAGE

You may want to include demographic questions on your survey that ask respondents about various aspects of their identity so that you may analyze the results in a way that compares different groups of respondents. When doing so, it is important that the way you ask demographic questions is inclusive and respectful. This guide covers best practices for inclusive language in survey questions.

TESTING THE SURVEY

It is useful to convene a small group from the population(s) you intend to survey to have them test the survey and survey questions. This can help gauge how the questions you have written will be perceived and answered, and whether the survey is too long or complicated.

SAMPLE VS. CENSUS

When planning a survey, it is important to decide if you will need to administer the survey to an entire population (all MIT seniors, for example), i.e. a census, or to a sample of the population (200 randomly-chosen MIT seniors). This choice can largely be informed by how you want to analyze the results. If you are interested in broad measures, such as overall satisfaction, a sample may suffice. If, however, you want to answer questions about students in Math versus students in Chemistry, you may need a larger sample or a census in order to net enough respondents for your analysis.

SMALL CELL SIZE

It is important to make sure that you do not report results for very small groups of people. For example, if your intended report would break out responses by department, but only 4 people responded from one department, then reporting those responses could jeopardize the privacy of those respondents’ answers. IR only reports summary results and uses the standard practice of only reporting cell sizes of 5 or greater.

REPRESENTATIVE SAMPLE SIZE

While a sample needs to be large enough to protect the privacy of the respondents (see above), it should also be large enough for your results to be considered representative. If you are interested in the responses of women in Math, and you only receive 5 responses from women in Math, (out of 20 women in Math, for example) you could not report with confidence that their summary responses as representative of all women in Math. To calculate the minimum sample size to be considered representative within a given margin of error, visit: http://www.surveysystem.com/sscalc.htm

KEEP IT SIMPLE

The simpler and shorter a survey, the more likely respondents are to answer all questions and answer them honestly. Longer, more complex surveys can annoy respondents and cause them to abandon the survey part-way or begin to pick arbitrary answers to speed to the end. Also, only ask questions that you need and intend to report.

ARCHIVING DATA

Once you have finished gathering your data, please consider archiving the survey instrument, administration details, and record-level responses with IR. IR maintains a secure server for MIT data and repots, and will be glad to archive data collected at MIT. This can be especially helpful, for example, when a student groups changes leadership. If that student group administers a survey and does not archive the data, the results can be lost instead of used to inform future decision making.