Scholarship Methods: Writing Surveys & Conducting Interviews
Learn about the differences between quantitative and qualitative data, research methods like surveys, interviews, and focus groups, principles of research design, and the purposes of quantitative and qualitative research. Understand how to approach each type of research and select appropriate data sources for your study.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Scholarship methods writing surveys, conducting interviews and running focus groups Diane Butler and Clem Herman , eSTEeM Directors 7th March 2019
Aims of session Overview 1. The differences between quantitative and qualitative data 2. Research methods including Surveys Interviews Focus groups Thanks to Dr Chrysoula Mangafa, Dr Duygu Bektik and Dr Laura Hills 12/04/2025
Principles of research design Research methods are split broadly into quantitative and qualitative methods Which you choose will mostly depend on your research questions Plus: your preferences and skills your underlying philosophy of research 12/04/2025 Acknowledgement Dr Chrysoula (Chrissy) Mangafa
Quantitative & Qualitative research Quantitative research aims to test hypotheses, look at cause and effect, and make predictions - answers questions such as How many? What? When? Qualitative research aims to understand and interpret social phenomena and interactions - answers questions such as Why? How? 12/04/2025
Quantitative approaches Attempt to explain phenomena by developing hypotheses and collecting numerical data Can be descriptive but usually analysed through statistical methods Tells you if there is a difference e.g. between conditions or groups More objective : where variables are controlled as much as possible to eliminate interference and measure the effect of any change Deductive reasoning: everything is known before conclusions can be drawn 12/04/2025
Sources of quantitative data Data sources include Surveys where there are a large number of respondents and where you have asked closed questions (e.g. use of a Likert scale, multiple choice, select answers from lists) Observations (counts of numbers and/or coding data into numbers) Analytics (number of views; task time etc.) Secondary data (institutional data; internal reports etc.) 12/04/2025
Qualitative approaches Instead of numerical data uses words, pictures, photos, videos, audio recordings. Tends to start with a broad question rather than a specific hypothesis Doesn t require large sample sizes (in comparison to quantitative research) More interpretive : yields rich data to explore how and why things happened Inductive reasoning: conclusions can be drawn from the evidence 12/04/2025
Sources of qualitative data Free text responses in surveys Interviews and focus groups (structured, semi- structured or unstructured) Diary studies Direct observations (face to face or online) may also be recorded (video/audio) Secondary data archive data and institutional reports 12/04/2025
Main differences between qual. and quant. research Quantitative Qualitative numerical, measurable data generally non-numerical data (words/images) clearly stated questions, extraneous variable controls exploratory questions, extraneous variables not controlled, large samples small sample size generalisability difficult to replicate or generalise experiments, RCTs, large surveys, measurements/instruments, questionnaires interviews, observations, focus groups, field notes, workshops role of researcher: detached, non-biased role of researcher: involved, personal beliefs influence the research statistical analysis interpretive and descriptive analysis final report: comparisons of means and statistical significance of findings final report: narrative report with contextual description &direct quotes from participants 12/04/2025
Mixed methods In reality most SoTL projects rely on a mixed methods approach which uses both quantitative and qualitative data collection and analysis in an effort to combine the advantages of both types of data. allows a triangulation of evidence and methods that provide a stronger logical case in drawing conclusions 12/04/2025
Activity your experience of different methods As participant or researcher Surveys Interviews Focus groups On a scale of 1 to 10 how comfortable are you with using each of these? 12/04/2025
Surveys OU Process - Student Research Project Panel (SRPP) SRPP Panel consists of representatives from the Faculties; Student Services; the Library; Marketing; Data Protection; and OUSA. SRPP Working Group. A subgroup of the main panel who meet once a fortnight to approve applications. The Student Research Project Panel (SRPP) was set up in order to ensure that Open University enquirers, students or alumni are not asked to participate in research that does not meet University guidelines. Where the guidelines refer to students , please take this to mean enquirers, current students and alumni. By reviewing applications and offering guidance to researchers, SRPP also fulfils an informal quality assurance function Anyone wishing to undertake research or scholarship with OU students (undergraduate and postgraduate) and alumni should apply to SRPP 12/04/2025 Acknowledgement Dr Laura Hills
Surveys Submitting an application Look at the guidance on the SRPP website: http://intranet6.open.ac.uk/mgt-info/iet-stats/srpp Contact the SRPP team if you have any questions Send the completed form to srpp@open.ac.uk by 4pm on the Thursday before the meeting. SRPP deadlines are fortnightly with SRPP meetings the week after the submission date. Don t forget to submit a Data Protection form and visit the HREC site at the same time as submitting the SRPP application. 12/04/2025
Surveys What makes a good SRPP application? The reviewing panel check the following The student sample request, including clashes with existing or planned research Will the proposed research instruments deliver the answers to the stated research questions? Is the information provided to students written in an accessible and friendly way? Is it sufficient for them to give informed consent ? Does the information make clear to the student the benefit of participating? Is there clear information about data protection? 12/04/2025
Surveys Designing and running a survey tips and hints The surveys team can either set up and administer your survey themselves using Qualtrics Or give you access to a user account on the Jisc (previously BOS) Online Surveys system The Surveys team will give you advice on the student sample you have asked for but you may find that there are fewer students available in your target groups to be surveyed than you hoped you may need a contingency for this. Alternatively you may which to survey students via your module website using Real-Time Student Feedback What ever system is used requires you to design a good survey i.e. one which works for users on all levels and addresses your research questions 12/04/2025
Surveys Designing and running a survey tips and hints (cont.) Think about Invitations and reminders How long your survey will be open for Timing your survey around module activities Asking for student demographic data and making it accessible in your survey tool Length of survey no of questions Conditionality of routing through your questions Balance of question types closed and open Types of closed question - variety while still retaining clarity! If you want to follow up respondents for interviews/focus groups ask for consent and emails addresses in your survey Getting your survey checked before it is released 12/04/2025
JISC Survey tool Demonstration https://www.onlinesurveys.ac.uk/help-support/ https://admin.onlinesurveys.ac.uk/account/openuniversit y/home/ 12/04/2025
INTERVIEWS AND FOCUS GROUPS Acknowledgement to Dr Chrysoula Mangafa Dr Duygu Bektik
Advantages vs. Disadvantages Focus groups + Allows for group discussion, interaction + More efficient collect data from multiple people - Need to coordinate and manage multiple participants - Subject to group dynamics (group effect) - Less anonymous (sensitive topics) Interviews + Allow for in-depth discussion with an individual + Easy to schedule + More suitable for sensitive topics - More time consuming than focus groups - Reactive effects influence of the interviewer
Types of Interviews Face-to-face Synchronous time and place Pick up on non-verbal cues Phone/Skype/Text-chat Synchronous time, asynchronous place Wider reach in terms of recruiting participants Email (Hamilton & Bowers, 2006) Asynchronous time, asynchronous place Allows participants to consider their answers before responding
Details to consider Length of session Interviews: 30 mins-1 hour Focus groups: 1-2 hours Number of participants Concept of saturation : where additional numbers don t provide any additional insights Rough estimate (for homogenous groups): Interviews: 10-15 Focus groups (ftf): 6-10 Focus groups (online): 4-8 Participants will drop out at the last minute
What data to collect Note taking Can be difficult to write notes and listen Useful to have an assigned note-taker in a focus group Recordings Audio recorders Video cameras In-built function (Skype or Adobe Connect) Will require transcription Chat logs and emails
Designing questions Structured, semi-structured and unstructured General advice Questions should relate to the project s aim Keep them simple and short Avoid the use of jargon e.g. unfamiliar acronyms, specialist terminology Don t have too many questions keep in mind how long the session will be Pilot your questions and refine them
Try to avoid Multiple questions e.g. How did you allocate the students to role play partners, prepare them for the exercise and ensure all students understood what was required of them? Leading questions e.g. Please explain the value of the role play exercise in helping the students to develop their patient care skills Yes or no questions e.g. Did you talk to your students about the role play exercise?
Types of questions Use pre-determined questions Identify potential questions in advance Use open-ended questions What did you think of the ? How did you feel about the ? What do you like best about the ? Use "think back" questions. Take people back to an experience and not forward to the future
Activity: Your project Developing interview questions Consider a scholarship project you are working on, or plan to work on What would you like to find out? Work in pairs to come up with 2-3 main questions that you could ask them (including some prompts and follow- up questions) Try out pre-determined, open ended and think back questions What would you expect them to say? What would these answers tell you?
Focus Groups Before the focus group Design, recruitment, logistics During the focus group Structure, being a skilful moderator After the focus group Transcription, analysis, reporting
Before the session Online vs. F2F How many sessions will be conducted? How many participants should be recruited? Logistics: arranging the location, room, equipment, drinks/snacks? How to prepare yourself skilfully as a moderator? What will you ask? Preparing the questions!
How many participants? Preferred number of carefully selected similar types of focus group participants is 6 to 8 (Krueger, 2002). Max. 10 people is suggested per session (Morgan, 1996; Krueger, 2002). o If it is online, make sure to have less: 4-5 How many focus groups? One or preferably more (especially if focus group is the only technique used in the project) Rule of thumb is 4 to 6 focus groups for projects with focus group technique only (Morgan, 1996).
Recruiting Participants Homogeneity vs. Heterogeneity: Focus groups can be organised as homogenous or heterogeneous Participants could be categorised as homogeneous as possible by age, sex, marital status, socioeconomic status, geographical location etc. Could have multiple focus groups with different set of homogenous groups. Ideal if focus groups is the sole method Eg: If you are looking at the effect of age on the use of social media, you might have several focus groups based on age, 20s, 40s, 60s etc. Homogeneous groups tend to be more willing to share their feelings eg women only groups Heterogeneous groups may be ideal if you don t plan to have several focus groups and use it in combination with other methods. Heterogeneous groups may spark more ideas, and you would collect different points of views in one go.
Logistics Location of the focus group session should be comfortable and participants preferably needs to be seated in circle setting to provoke interaction. Face-to-face Need to arrange a large enough room Room setting: Not a theatre but more of a cabaret style Circular, see each other and discuss Online (e.g. Skype or Adobe Connect) Wider recruitment reach (no need for traveling to campus) Quality playback, equipment for recording Drinks/snacks!
Moderator(s) Lead and Assistant moderator(s): sessions are moderated by the researcher who runs the discussions and assistant moderator who will not participate in the discussion but arrange logistics such as helping with the equipment, refreshments, arranging the room, recording the session etc. (Krueger, 2002). Most importantly, takes notes!
Beginning Welcome Introduce yourselves Overview Topic is Will be used to inform You were selected because Ground rules No right/wrong answers, only differing views No need to agree others etc. Tape/video recording and first opening question!
Dos! Do listen for inconsistent, vague and cryptic comments and probe for understanding: "Would you explain further?" "Would you give an example?" "I don't understand. Control YOUR reactions to participants and AVOID Head nodding, Affirmative hmms, Short verbal responses: "that's good", "excellent" Why? Because we don t want to impose our thoughts and lose the participants who might not necessarily agree. Participants are allowed to say anything they d like to which makes focus groups considered as naturalistic. Ensure everyone has chance to participate control the dominant talkers and encourage the shy participants
Ending Consider asking each participant a final preference question Offer a summary key question and seek confirmation. Three Step Conclusion: Summarise with confirmation, Review purpose and ask if anything has been missed, Thanks and dismissal ACTIVITY what s working in this Focus Group? https://youtu.be/Auf9pkuCc8k
