Loading...
HomeMy WebLinkAboutWORK ORDER - RFP - 7226 SURVEY SERVICES - MISC PUBLIC OPINION SURVEYSWORK ORDER FORM PURSUANT TO AN AGREEMENT BETWEEN THE CITY OF FORT COLLINS AND National Research Center DATED: 5/26/2011 RFP Work Order Number: 7226%� Purchase Order Number: 32 51 Project Title: Natural Areas Survey and Visitor Count Commencement Date: May 26, 2011 C Completion Date: August 31, 2012 Maximum Fee:4 J-y 60-d . d Project Description: Survey residents about their perceptions of the Natural Areas Program and count the number of visitors to each site. Scope of Services: Attached. Service Provider agrees to perform the services identified above and on the attached forms in accordance with the terms and conditions contained herein and in the Services Agreement between the parties. In the event of a conflict between or ambiguity in the terms of the Services Agreement and this work order (including the attached forms) the Services Agreement shall control. The attached forms consisting of 6vG ( E� ) pages are hereby accepted and incorporated herein by this reference, and Notice to Proceed is hereby give. National Research Center, Inc.: By:� Q Date: S l- ao I I CC: Purchasing City of Fort Collins: 1 Submitted by: _Zoe Whyman Project Manager Reviewed By: Date: 5/26/11 Date: Approved by: G� N� ural Areas Program Mgr. Date: Approved by: Director of Purchasing and Risk Management (if over $60,000.00) Date: City of Fort Collins Natural Areas Program Survey and Visitor Count Scope of Work for 2011 Natural Areas Survey 1) Implement a visitor count study Work with NAP to understand the complexities of the natural areas and their points of entry, develop a visitor count protocol to capture seasonal variation in visitation. We will work with NAP to devise the observation schedule and data collection protocols. We will work with NAP to understand all the various areas to be included, what the data entry points are, and any issues that may arise during observation. We will help to devise a data collection schedule to allow the most efficient use of observation hours and other tools such trail/ vehicle counter and other previously collected data. We will want to capture the most data, but to ensure representation from both high and low use areas. Help with training on visitor count study. We will work with NAP to train those collecting the data, but our organization will not do the data collection. Tabulate the visitor counts, calculate estimates of total visitation, summarize the results and methods in the final report. Provide a spreadsheet of the count data to NAP. We will analyze and summarize the results and methods. We will use the available data to create the most accurate estimates possible of total visitation, and visitation by site and season. 2) Implement a resident survey Task 1: Development of Survey Questions and Mailing Materials It is our experience that, to achieve the best results, time must be invested up -front to craft the right survey instrument and to tie survey design to the overall goals and objectives of the project. We regard the questionnaire development process as an iterative one, with drafts being reviewed and revised by a project oversight committee or point person and our team. NRC will work with the City to develop a comprehensive questionnaire that covers the objectives of the study. We will start with the 2006 visitor survey instrument and the list of desired modifications/additions from the RFP to draft the questionnaire. We will work with the City in an iterative process until a final questionnaire is developed that is approved by everyone. The survey will be 3 pages in length. Task 2: Sampling Selecting survey recipients In survey research, it is desirable for every household (or every adult person living in a household) to have a known and equal chance of participating in the survey. A sample of all households is selected to be given an opportunity to participate. Sampling is based on probability theory; the idea that if respondents are chosen randomly from the larger population, the results from that random sample will be very close to what we would get by interviewing every member of the population. The challenge is to find a list of all households from which survey recipients can be randomly chosen. For a mailed survey, mailing lists exist from the United States Postal Office which include all households with a mailing address. All households located within the boundaries the City determines are appropriate for this study would be eligible for the survey. We will geocode the location of each address to assure it is within the study boundaries (as well as any desired subareas within the overall boundaries, if desired). National Research Center, Inc. e e City of Fort Collins Natural Areas Program Survey and Visitor Count We will use systematic sampling from the list of eligible households to select the sample of households. Systematic sampling is a procedure whereby a complete list of all possible addresses is culled, selecting every Nth one until the appropriate number of households is sampled. Not only will NRC scientifically and randomly sample households,.but we also will select without bias the household member to participate. In cases where more than one household member is 18 or older, a procedure called "the birthday method" will be used to select the survey respondent without bias. The birthday method requests that the respondent be the eligible adult in the household who most recently had a birthday (irrespective of the year of birth). This methodology helps ensure the attitudes expressed by our respondent "sample" closely approximate the attitudes of all adult residents living in the study area. Without this step in survey research, it is likely that results will be biased towards those who are more sedentary and those without jobs (who may have different opinions about some services). Choosing the number of completed surveys ("sample size") The relationship between sample size and precision of estimates (the 95% confidence interval or margin of error) is shown in the adjacent table. The margin of error or 95% confidence interval is determined solely by the number of completed surveys, and not the size of the . population from which the sample is drawn, as long as the "population" size is greater than about 4,000. The table can be interpreted as follows: If 61% of 400 respondents said they would support a mandatory vehicle emissions program in Fort Collins even though it is not Federally required, we can be confident that, had we gotten ratings from all residents, somewhere between 56% and 66% would have supported mandatory vehicle emissions. We will send surveys to 1,600 households, for a likely final completed sample size between 400 and 600, which would provide a 95% confidence interval of plus or minus 5 percent. � Sample ' ,-_Size Mf argin:oError"��, (95q Confidence4nterval) \. 100 - ±9.8% zoo ±6.9% 400 +4 9% 600 ±4.0% 800 ±3.5% 11200 ±2.8% 2,000 ±2.2% Task 3: Survey Administration Below is outlined our survey administration procedures. We encourage our clients to inform the public about the survey through local newspapers, newsletters, community meetings and any other appropriate media. The publicity provides an opportunity to inform them about the purposes of the survey and the uses - for programming and policy - to which it will be put. The publicity, furthermore, will assist the survey process by increasing willingness of potential respondents to participate. Mailed Survey Data Collection As described above, a sample of addresses from within Fort Collins and/or Larimer County will be selected to be part of the survey. This database of addresses will be processed for certification and verification. NRC vendors use CASS"/NCOA software that relies on the USPS National Directory information to verify and standardize the address elements and assign each a complete, nine -digit zip code where possible. In addition, the software will sort and barcode the addresses, allowing significant postage discounts. We propose mailing all materials pre-sorted first class. This ensures a fast delivery time of the materials, but provides an opportunity for a discount on the full first-class postage rates. Our mailhouse will oversee the printing and preparation of the prenotification postcards, survey packets and reminder postcards. As a part of the quality control process, proofs of the survey materials are reviewed by NRC staff before the final job is printed. We will include a point person at Fort Collins as a "seed" in the mailing list, so they will receive the mailing materials at the same time as the sampled recipients. National Research Center, Inc. City of Fort Collins Natural Areas Program Survey and Visitor Count We propose three contacts with each sampled household: 1) A prenotification announcement, informing the household members that they have been selected to participate in the community survey will be sent to each household. This announcement should be scheduled to arrive about a week before the survey packet. 2) Approximately one week after mailing the prenotification, each household would be mailed a survey containing a cover letter (ideally signed by the mayor or other appropriate high-level official or staff member) enlisting participation. The packet would also contain a postage paid return envelope that would send all completed surveys directly for data entry. 3) A reminder letter and survey, will be scheduled to arrive 1 week after the first survey. The second cover letter asks those who have not completed the survey to do so and those who have already done so to refrain from turning in another survey. We mail the survey twice because anonymity is promised in the cover letter to enhance the likelihood of honest responses. We take this implicit contract with respondents as a serious principle of the survey trade, which, if violated, harms the survey research industry no less than the client or respondent. We can also check for duplicates if we include a survey code for later mapping of responses. Generally we find only one or two duplicates in our responses. When the survey administration mode has been chosen, a sample size selected, and the survey materials finalized, NRC will produce a memo for the City outlining the survey protocols and containing the final survey materials. Task 4: Data Tabulation Survey processing Mailed surveys will be returned to an NRC affiliate directly via postage -paid business reply envelopes where staff will assign a unique identification number to each survey. Additionally, each survey will be reviewed and "cleaned" as necessary. For example, a question may have asked a respondent to pick two items out of a list of five, but the respondent checked three; NRC staff would choose randomly two of the three selected items to be coded in the dataset. We have found that very little cleaning is needed on most surveys due to our expertise in question construction and survey formatting. However, extensive cleaning tends to be needed in surveys containing complicated question structure or skip patterns. Once all surveys have been assigned a unique identification number, they will be entered into an electronic dataset. This dataset will be subject to a data entry protocol of "key and verify," in which survey data are entered twice into an electronic dataset and then compared. Discrepancies are evaluated against the original survey form and corrected. Range checks as well as other forms of quality control will also be performed. Weighting the Data The last step in.preparing the data for analysis is to weight the data to reflect the demographic profile of the Fort Collins/Larimer County population. Weighting is an important method to adjust for potential non - response bias. In general, residents with certain characteristics (for example: those who are younger or rent their homes) are less likely to participate in surveying, whatever the data collection mode. Weighting allows us to look at the demographic profile of residents who returned the survey compared with the US Census profile. We consider these disparities along with others and accordingly increase or decrease the weight of each respondent to mimic as closely as possible the population demographics. The weighting variables to be considered will be all those demographics included on the survey, as well as any geographic variable used for sampling. NRC has extensive experience with complex weighting schemes required with stratification. National Research Center, Inc. City of Fort Collins Natural Areas Program Survey and Visitor Count Basics of Data Analysis and Reporting It is paramount that the result of this study be more than a nice looking report that sits on a shelf, and as such we will work with the City of Fort Collins to ensure that we ask the right questions to feed into an actionable analysis plan. For quantitative analysis, we rely on the Statistical Package for the Social Sciences (SPSS). We believe that analysis must be replicable and leave a clear path. To this end, we keep every label and command run in SPSS in a syntax file available for audit and re -running, as necessary. We also have trained clients on SPSS analysis and, for small recurring analyses, how to use Microsoft® Excel. We will code open-ended responses using both an emergent approach, where themes are revealed through the analysis, combined with a deductive approach, where a scheme or codes are predetermined and applied to the data. Techniques include word counts, content analysis, comparative analysis, componential analysis and taxonomic analysis, among others and codes are applied based on the unit of analysis. Our qualitative analysis of verbatim responses will be done using Microsoft® Access, Microsoft® Excel or QSR qualitative software, depending on the types and complexity of the open-ended questions. The data and report will undergo a thorough quality assurance review. We will audit the original data files, our syntax/analysis files, compare automatically generated output to the formatted output in the report and data check all numbers and text prior to submitting the report. We will provide basic descriptive statistics such as frequencies and means summarizing the responses to the survey questions. If it makes sense to create scales or indices that summarize information from a number of survey items, we can use factor analysis and other psychometric testing to create valid factors. Additionally, NRC staff often have employed other advanced data analysis techniques to transportation survey findings to determine what the key drivers of respondent satisfactions are (using linear or logistic regression to demonstrate the factors that are most important in respondent satisfaction), cluster analysis (market segmentation) to identify groupings of respondents whose members share characteristics but whose clusters differ in important way. For example, in a study we did for the Regional Air Quality Council on ground level ozone pollution, we found there were four clusters or segments of the population in regards to attitudes towards and behaviors to ameliorate ground level ozone, ranging from the "Negative Normans" (comprising 12% of the population) who did not feel ground level ozone was a problem and were unlikely to make any behavior changes to decrease ground level ozone to "The Eager" (comprising 22% of the population) who reported already making lifestyle changes to reduce ground -level ozone pollution and who were ready to be influenced to change behavior. Upon completion of the data analysis, NRC can provide an electronic copy of the dataset to the City of Fort Collins in a variety of formats. Typically, clients prefer to receive the data as an Excel worksheet, but we can also provide an ASCU dataset, an Access dataset, or an SPSS dataset. We will provide information about the layout and value labels of the data so that you will have no trouble accessing the raw data. As a note of caution: while we can provide the dataset in Excel, including the appropriate weight for each case, applying the weights in Excel is quite difficult. SPSS or another statistical application maybe required to apply the statistical weights and derive the identical results to what would be displayed in the report of results. However, we keep all the data files on our own server, and will be happy to run additional crosstabulations for you (at no additional cost) as questions come up as you use and disseminate the report. Task 5: Reporting the Results Report of Results NRC understands that most readers want to understand the broader patterns and interpretations of data, so our professionally designed graphics are engaging and pop with clear information. Our reports are comprehensive and include technical and detailed numbers and information, but not at the expense of the National Research Center, Inc. .,May 20"2011 City of Fort Collins Natural Areas Program Survey and Visitor Count day-to-day usefulness of the documents. We are careful about our communication. You will not need a degree in statistics to understand the survey results. All the technical details are in appendices for those who wish to read them (and so that the survey can be repeated by any vendor you choose), but we make sure the report body tells the story of the survey results in a stylish, colorful, informative way. For further ease, we also include an executive summary that gives a quick overview of results while highlighting key findings. We will prepare the report in Microsoft® Word (as well as convert documents into a PDF format). Drafts of the report will be provided to the City and the final report will incorporate any comments we receive from you. We will meet with you to review the draft report, and answer any questions or comments you have. We will provide an electronic copy of the report in Microsoft Office Word® format along with a version in PDF format (readable by Adobe Acrobat). We will also provide a hard copy. of the report. . Consulting and Continued Support ' We know that an effective report will be reviewed and discussed by many.people in Fort Collins after we have completed the survey projects, and that new questions may arise. We commit to supporting these discussions by offering crosstabulations and other additional analyses of results when needed. And, though we will not abandon you with your data, we always will provide you with the electronic dataset of your responses for your future reference and use. National Research Center, Inc.