Developing good survey questions can be a difficult process, but by putting the time and effort in at this stage, you can ensure that the data you collect will be worthwhile and help you get solid answers to your research questions. There are few things worse in the research process than doing lots of work to conduct a survey, only to find out at the end of the process that your questions did not really elicit the types of responses you expected.
Get participatory! The process of constructing relevant questions is one of the best ways to get community members involved in the research process.
Before you begin the main portion of the survey, you might want to think about asking a few “screening questions” to make sure that the person taking the survey is the type of person you want to get information from. These questions might be about their age, race, religion, years living in an area, an institutional affiliation, or any number of other issues. For example, the Metamorphosis Project conducted work in the LA County city of Alhambra. We wanted to speak with adults who had lived in the community for at least two years in order to get the opinions of longer-term residents of the area. Here was our screening procedure:
Another early consideration is whether you want open-ended or close-ended responses to your questions. Open-ended questions leave the respondent room to answer as they please – their response could be a word, a phrase, a paragraph or more, depending on how the survey guides them. Closed-ended responses offer a few possible responses that respondents can choose from. Many surveys use a combination of both.
Below are some examples from the Metamorphosis Project. Often, close-ended questions ask participants to respond to a statement and place their answer on a scale, like the examples below that come from our measure of neighborhood belonging:
By using this type of numerical scale, we can easily quantify the information and later analyze it using statistical techniques. If you are interested in conducting statistical analysis, you might want to use a scale – like the neighborhood belonging measure from Metamorphosis – that have been used successfully by other researchers in the past. This way you can be sure that the questions have been well-designed and you can also compare your data to those of other researchers.
There are several types of scales that can be used in survey questions.
Rating Scales (often called Likert Scales), like the one above, ask the respondent to rate a statement, usually on a 3-point, 5-point or 7-point scale (more than that is too much). Likert scales can ask about things like agreement, frequency (very frequently to never), importance (very important to not at all important), quality (very good to very poor), likelihood (very likely to not likely at all), or other topics.
Ranking Scales might ask respondents to put a few options in rank order. For instance, the Metamorphosis Project often asks respondents to rank the top two types of media they use to stay on top of community issues (like local television, radio, or the newspaper).
Here are a few open-ended examples:
These answers are still up to the respondent, but there is a chance that these responses could be several sentences or even a paragraph or more, unlike in the closed-ended examples. While this provides valuable information, it may be difficult to quantify their responses – that is, put them into numbers that can be analyzed with statistics. This type of qualitative information provides important but different types of research data.
There are a number of other considerations that must come into account as you design the content of your survey. Surveys should go through different revisions, when possible, and have several people work on and revise them to make sure the questions are appropriate and easy to understand. Once again, if you get participatory you can avoid a lot of the mistakes researchers often make in this area. While we cannot go through ever single issue that should be thought about in the development of the survey, here are a few important ones to consider:
How personal will your questions be?
How do I avoid socially desirable answers?
How detailed or complex will your questions be?
How long should the survey be?
Does question order and sequence matter?
Again, these are some but not all of the concerns that should be in mind when developing the survey itself. Please see our additional resources for other in-depth treatments of survey design.