Actively engaging adolescents in meaningful program evaluation is a topic of growing interest. One possibility for such engagement is the use of photographs as part of visual evaluation, so that youth can directly engage with the research process. In this Method Note, we describe the development and implementation of a participatory, photo-based evaluation method for youth health promotion/prevention programs. Youth in this study were participants in a gender-transformative healthy relationships program for boys. We present literature supporting the use of photographs as a visual research method and for involving youth as active participants in evaluation, and explore the feasibility, utility, and acceptability of this innovative application of existing methods based on researcher experience and youth feedback. We conclude with implications for photo-based evaluation of health promotion/prevention programs, highlighting the promise of this method for promoting critical youth engagement in evaluation and the creation of meaningful knowledge translation tools.
Current research with adolescents suggests that traditional “adult-centered” qualitative methods (e.g., interviews) are not optimal for enabling a rich understanding of youth perspective and experience (Hieftje et al., 2014). Instead, visual research is an emerging method that allows researchers to authentically engage youth in program evaluation (Strack et al., 2004). In visual research, participants use visual media, such as photographs, to identify changes experienced in the program and to then share these changes with policy makers and other key stakeholders (Halsall & Forneris, 2016; Kramer et al., 2013; Raber et al., 2016). Such evaluation strategies also enable youth to become partners in evaluation (Checkoway & Richards-Schuster, 2003). However, very limited research that explores the use of photography for youth participatory evaluation in the context of health promotion/prevention programming exists. Thus, the purpose of this project was to develop and explore the use of a participatory, photo-based evaluation method for health promotion/prevention programming with adolescents.
Photographs as a Visual Research Method
The use of visual research is an increasingly popular method in the social sciences. For participant-based photography as a visual medium in particular, the field is grounded in the seminal work of Wang and Burris (1997), who described a participatory action research method called photovoice. In photovoice methodology, participants are invited to be partners in the research project and use photographs to (a) reflect on a community’s needs and assets, (b) engage in critical dialogue about these strengths and challenges through group discussion, and (c) translate this dialogue to policy makers through presentation of photographs in the community (Wang & Burris, 1997). Photovoice is informed by feminist empowerment theory, Freirean pedagogy, and documentary photography (Wang & Burris, 1997). In regard to feminist theory, Wang and Burris sought a method that encouraged active participation and supported individual and collective empowerment. The ongoing cycles of dialogue, reflection, and action present in photovoice methodology are further reflected in Freire’s work (1970/2008). Finally, the history of using photography as a tool in advocacy and social change efforts is well-documented (Szto, 2008). Although originally described as a needs assessment strategy, the developers of the photovoice method also highlighted its potential for facilitating participatory evaluation (Wang, 1999; Wang & Burris, 1997). Particularly, Wang (1999) describes that “photovoice enables participants to document the success and failures of program activities and interventions. It can vivify people’s views of process and impact” (p. 190).
Though early photovoice research used disposable cameras, the vehicle for picture-taking continues to evolve with technological advances, so that the method is “creatively and flexibly adapted to the needs of its users” (Wang & Burris, 1997, p. 383). For example, Yi-Frazier and colleagues (2015) adapted the photovoice method by using social media (Instagram feeds) as the source of photos in a study exploring the experience of type 1 diabetes with adolescents. Common to all applications of photovoice, however, is the focus on participatory research toward social change through individual and community action (Wang, 1999). Wang (2006) also specifically discusses how photovoice can be used in youth participatory research and principles for the use of photovoice with youth (i.e., involving youth in all aspects of the research; building a co-learning, reflective, and enabling process; and balancing the goals of research, action, and evaluation).
While photovoice has gained particular prominence in the last 20 years, there are also a number of other photo-based methods. For example, photo-elicitation involves using photos (or other images) as part of interviews (Harper, 2002); photos may be taken by participants (e.g., Oliffe & Bottorff, 2007) or provided by researchers to participants (e.g., Epstein et al., 2006). Photo-elicitation may be a particularly appropriate method for work with children and youth as the use of photos as the basis for discussion can scaffold deeper sharing of thoughts, feelings, and perspectives (Ford et al., 2017). Visual storytelling is an example of photo-elicitation, with the goal of actively engaging youth in the research process (Drew et al., 2010). In this method, youth participants take photographs in response to a series of prompts and then participate in a participant-led, individual interview where they describe their photos. Drew et al. (2010) developed this method to explore experiences of chronic disease self-management with adolescents.
Given its roots in participatory action research and empowerment, photovoice is an excellent method for conducting participatory research (Sitter, 2017), and emerging research in this area also engages youth. For example, Strack et al. (2004) used this method to explore needs and assets for youth in an urban after-school program. Youth in their study were satisfied with the overall process of photo-taking and felt empowered after participating. Based on their experiences, Strack et al. (2004) provide a suggested 20-session curriculum for conducting photovoice projects with youth.
Prior Visual Evaluation Work With Youth
We are aware of five prior studies using photo-based methods as an evaluation tool (Halsall & Forneris, 2016; Kramer et al., 2013; Raber et al., 2016; Seed, 2016; Skrzypiec et al., 2013), but only one of these specifically focused on youth (Halsall & Forneris, 2016). In this study, Halsall and Forneris (2016) used youth participatory evaluation with Indigenous Canadian youth leaders (age range 12–18) to understand participant experiences at a youth-led community event. Youth in the project received training on photography techniques and photovoice ethics. Photos were then taken during the community event and subsequently explored in a workshop format. Though not exclusively with youth (44 adults, 6 youth in the sample), Kramer et al. (2013) used photovoice as a retrospective evaluation method, so that participants could identify “key changes that they felt had occurred in their communities as a result of the initiative” (p. 687). In this project, photos were taken before the intervention (an organization-level community health improvement initiative focused on healthy eating and active living) to describe what changes the community was looking for on these topics (i.e., more of a needs assessment) and after to document changes that had occurred in the community since the intervention. As in the study by Halsall and Forneris (2016), participants in the study by Kramer et al (2013) received training on photovoice ethics. Each participant in the study by Kramer et al. (2013) discussed their top five “after” photos using the SHOWeD method, a common method for facilitating photovoice discussion (e.g., Wang, 2006). However, to our knowledge, no prior studies have used photos as the basis for the evaluation of health promotion and prevention programming with youth.
The Current Project
The first author is currently engaged as the lead outcome evaluator for a gender-transformative,1 community-facilitated, healthy relationships program called WiseGuyz. WiseGuyz was developed by the Centre for Sexuality and is offered to mid-adolescent, male-identified participants through a school–community partnership model. The goal of the program is to improve mental and sexual health and reduce male-perpetrated violence (for more details on the program, see Claussen, 2019; Exner-Cortens et al., 2019). Prior quantitative outcome evaluation has demonstrated the promise of this program for improving the health and well-being of adolescent boys (Claussen, 2017; Exner-Cortens et al., 2019). However, beyond quantitative data collection, the research team wished to engage youth in participatory evaluation in line with the empowerment and antioppressive foundations of the program. In discussing the medium for this evaluation, the research team felt photographs were most appropriate, both due to the team’s prior use of this method (Sitter, 2017; Sitter, 2018) and because previous research has identified that photographic methods are useful when trying to understand gender as a social determinant of health (e.g., Haines-Saah & Oliffe, 2012).
In addition, exploring gender and health was one of the original applications of photovoice (Wang, 1999), and the use of photos as the basis for qualitative research may be particularly important among male-identified participants, who, because of social gender norms emphasizing emotional restriction and avoidance of vulnerability, may have difficulty expressing thoughts, feelings, and emotions in a typical semistructured interview format (Affleck et al., 2012). Given that our participants sit at the intersection of youth and male gender—both groups for whom semistructured interviews may pose challenges (Affleck et al., 2012; Hieftje et al., 2014)—the use of photo-based methods seemed especially appropriate. As we did not locate any existing work that used photo-based youth participatory evaluation to understand changes in a health promotion or prevention program, we decided to combine existing approaches (namely, photovoice and visual storytelling) to create an innovative method for our evaluation (see Table 1 for method comparison). The purpose of this article is to present the method we developed, including adult and youth reflections on acceptability, feasibility, and utility.
Focus on social change and participant empowerment
X
X
Photos as basis for group discussion
X
X
Photos as basis for individual discussion
X
X
More feasible for a shorter number of interactions a
X
X
Integration of group-based and individual-based methods
X
Designed as a program evaluation tool
X
a This is facilitated in part as the photo-based evaluation method is designed for use with existing groups as part of group-based youth health promotion/prevention programs.
Method Overview
Reviewing the photo-based literature, we felt that both the focus on group discussion, advocacy, and change found in photovoice (Wang & Burris, 1997) and the participant-led individual interviews used in visual storytelling (Drew et al., 2010) were features that were desirable for our project and together would give us rich data on the program as experienced by participants. Further, we felt these methods were appropriate for a participatory evaluation project with youth researchers. Thus, we chose to use a combination of photovoice and visual storytelling as our evaluation methodology in this project. The adaptation and combination of these methods was also required to meet constraints on participant engagement in the school setting in which our youth were located. Particularly, while many photovoice projects meet with participants numerous times over the course of the study (e.g., Strack et al., 2004), our location in the school setting meant meeting with youth photographers a minimal number of times over the course of the project in order to accommodate school staff schedules. Reviewing the photovoice and visual storytelling literature (Catalani & Minkler, 2010; Drew et al., 2010), we felt that meeting with youth photographers 3 times over the course of the project was a level of engagement that balanced methodology with context needs (Table 2). We applied our methodology with one school in a Western Canadian province; additional details on school selection are given below as part of understanding feasibility. This project was reviewed and approved by a university research ethics board and the participating school/school division.
Table 2. Overview of Youth Engagement.
Date
Purpose
Format
Topics Covered
February 2018
Photovoice training
Group
•
Icebreaker/group warm-up (topic: using photos to express emotion)
•
Project overview
•
Photovoice ethics
•
Photography 101
•
Transferring photos
•
Debrief
April 2018
Visual storytelling interview
Individual
•
Discussing top five photos
May 2018
Photovoice focus group
Group
•
Ground rules for sharing
•
Sharing photo stories (before and after photos)
•
Creating titles, narratives, and photovoice posters
•
Debrief
May 2018
Community presentation #1
Group
•
Presenting photovoice posters
June 2018
Community presentation #2
Group
•
Presenting photovoice posters
Youth Recruitment and Engagement
To recruit youth photographers, we asked WiseGuyz program facilitators at the involved school to inform all youth currently participating in the program about the evaluation project using a standard recruitment script. In the recruitment script, youth were told that as a partner in the research, their role in the evaluation would be to take photos to show how the program was working for them and then co-lead an individual interview and group meeting to explore their photos. As a research partner, they would also have the opportunity to lead the presentation of research findings at two community meetings. Facilitators gave all interested youth an information packet that contained a parent information letter, a parent consent form, a youth assent form, and a youth information form. Consent and assent forms were returned in a sealed envelope to the facilitator, so that the facilitator did not know the youth’s decision regarding participation. Youth and parents were informed as part of the consent process that the individual visual storytelling interview and photovoice focus group (see below) would be audiotaped and transcribed. The consent and assent forms also informed parents and youth that youth could withdraw their photos at any point by contacting the research team, but that if images had been distributed as part of any presentations, publications, or reports, it would not be possible to withdraw them. Finally, they were told that although names and other identifying information would not be used in the final project report or results dissemination, youth could choose to be identified as part of two community presentations, if they so wished. On the assent form, youth indicated if they wanted to be identified by their real name or a pseudonym during these presentations or if they did not want to participate in the community presentations. The youth information form asked youth to briefly describe why they were interested in participating, to confirm their availability for the two project group meetings, and if they had a digital camera or other way to take photos. If they did not, they were informed that the study team would provide one; however, all youth in our study had access to a camera.
Eight youth signed up to be part of this project, and six completed the project. Our initial goal was 7–12 participants. This was based on recommended group size in photovoice projects, which is kept small to allow for meaningful and safe discussion (Wang, 1999, p. 187). Had more than 12 signed up, youth were told that participation decisions would be made by random draw. We met with these youth up to five times over the course of the project (Table 2). These youth were all in the ninth grade, mean (SD) age = 14.40 (.31), 83.3% White, and from a rural area of a Western Canadian province. To honor their role as project photographers and researchers, youth received a $25 CDN gift card for each event they attended (Table 2), for a possible total of $125 CDN over the course of the project. The amount of $25 CDN per event was based on the minimum wage in the province where the evaluation took place. Parents were also reimbursed for mileage, and dinner was provided at both group meetings (Table 2).
Procedures
Following recruitment, we held our first group meeting (Table 2). At this meeting, youth participated in a group warm-up led by a WiseGuyz facilitator from a different school. This facilitator created a warm-up activity that focused on using photos to express emotions. The facilitator, an active photographer, brought in photos he had taken and asked the youth to interpret these photos (e.g., What feelings do the photos elicit?) and then discuss with the group. Following this warm-up, the first author gave an overview of the project (e.g., purpose of project, time line, key study questions). As part of this project, youth participants were asked to take pictures that addressed two key questions: What does it mean to be a guy in your world (1) before WiseGuyz? and (2) after WiseGuyz?2 The use of two focused questions is supported by previous pilot work on visual storytelling (Drew et al., 2010). The second author then led a discussion about the ethics of picture-taking, which were also informed by our location in a school: of particular importance was not taking photos on school property and not taking identifying photos of other people. The second author also discussed how to take high-quality photos. To transfer photos to the research team, youth used an app called WeTransfer—this app allows for quick and secure sharing of large files (up to 2 GB), so that photos are not compressed during transfer. Plain-language, youth-friendly handouts were also created by the third author, summarizing information covered at the training session (Figure 1).
Figure 1. Group meeting #1 (photovoice training) handouts.
Once photos were received, a project graduate research assistant worked with the school and the youth photographer to schedule a ∼30-min individual visual storytelling interview to discuss the youth’s five most meaningful photos (Kramer et al., 2013; the average interview length was 30 minutes, with a range of 15-51 minutes), per visual storytelling methodology (Drew et al., 2010; Table 2). Interviews were conducted by either the first or second author. Youth photographers led the discussion by talking about their photos and offering insight into what was meaningful for them. However, a brief semistructured interview guide was also prepared to prompt additional reflection (e.g., What do these images tell us about what WiseGuyz means to you? Of these five images, which is your favorite and why?). During the interviews, the youth photographer selected their favorite before and favorite after photos (i.e., top two photos). These photos were used for the photovoice focus group (Table 2). All interviews were audio-recorded and transcribed verbatim. Transcripts were analyzed using qualitative description methodology (Sandelowski, 2000) and following the recommendations of Drew and Guillemin (2014). Briefly, the first, third, and fourth authors first read all transcripts and met to create an initial codebook. The third and fourth authors then refined the codebook and established inter-rater reliability using an iterative process that included four rounds of test coding and team discussion. At the end of this process, inter-rater reliability (as assessed by Cohen’s pooled κ) was .83, which indicates very good agreement (Landis & Koch, 1977). Following reliability establishment, the transcripts were all coded by both research assistants who met to discuss each transcript after coding and came to consensus on any discrepancies. Once data were coded, the full research team met to review the codes and theme the data.
Following the individual interviews, we held a second group meeting (Table 2). The purpose of this meeting was for youth to engage in critical dialogue about each other’s photographs and then work to (a) select photo(s) that they felt best reflected their perceived experiences in the program, (b) create stories to describe what the selected photograph(s) meant, and (c) identify themes that emerged from the created stories (Wang & Burris, 1997). Prior to the focus group, the research team printed all of the youths’ photos for them, so that they had something to show the group and in case they changed their mind about which photo(s) they wanted to include in the photovoice poster display during the discussion. In the first part of this meeting, youth were coleaders of a 60-min photovoice focus group. During this time, youth photographers discussed their top two photos, as identified in the individual interview. To facilitate this co-led, collective discussion, we used an adapted version of the SHOWeD method (Wang & Redwood-Jones, 2001), asking youth to reflect on (a) why they picked these photos, (b) what is happening in these photos, and (c) how the photos relate to their time in WiseGuyz. Adult facilitators helped to guide the discussion by asking youth how their ideas related to each other and what themes they saw. To close the discussion, the research team asked the group who needed to see the photos and what it was that people needed to know. To facilitate further analysis, the focus group portion of this meeting was audio-recorded and transcribed verbatim. We used the same qualitative analysis methods as described above (for the interviews) for the focus group transcript.
In the second half of the focus group meeting, youth photographers worked individually to create their photo narratives and titles and then worked as a group to create the photovoice poster display.3 A priori, the research team decided to have youth create two posters: one on experiences of being a guy in the world before WiseGuyz and one on experiences after WiseGuyz. This decision was made as youth had only one hour to create the titles, narratives, and displays, and we felt that this constraint would allow the youth to complete the project in the allotted time. The youth led the creation of the posters by taping their chosen photo(s), titles, and narratives in the arrangement they wanted on poster board; the third author then created digital versions of the posters using PowerPoint.
The photovoice poster display was then shared at two community presentations with key stakeholders: the WiseGuyz graduation ceremony (attended by all youth WiseGuyz participants from across the region) and the Centre for Sexuality annual general meeting (attended by local not-for-profit organizations, policy makers, and community members). Youth photographers were given the option to present their work at both of these events; five attended the graduation ceremony and two attended the annual general meeting. All youth but one chose to be identified by their real name at the two community presentations.
Level of Youth Participation
As described by Checkoway and Richards-Schuster (2003), youth participation in evaluation comprises different levels of engagement and power-sharing, from youth as subjects (where youth have no input into the evaluation process) to youth as directors (where youth initiate and take a lead on all stages of the evaluation process). In addition, we acknowledge that the level of youth participation is not a fixed state but changes throughout a project. Overall, however, our methodology best aligns with youth as partners, where adults “initiate and implement the process, but enlist youth to assist them” and youth “assist adults in roles such as information collection and dissemination of findings,” in order to “develop [the] skills of young people and include youth voice” (p. 25). In our process, youth are the sole collectors of the primary data source (photographs), lead the discussion of this data source as part of secondary data collection methods (visual storytelling interviews and photovoice focus group), create a key knowledge translation document (photovoice poster display), and have the option to support knowledge translation at community events. Our participatory methodology can also be described as a youth–adult evaluation partnership (Zeldin et al., 2012; Checkoway & Richards-Schuster, 2003).
Feasibility, Acceptability, and Utility
To assess the pilot application of this innovative evaluation methodology, we chose to focus on its feasibility, acceptability, and utility (Kistin & Silverstein, 2015; Rounsaville et al., 2001). By feasibility, we mean the practical ability to implement this methodology within the typical constraints of applied community research (e.g., resources, time). By acceptability, we mean how youth and community research partners viewed the methodology (e.g., Were youth willing to sign up? Did youth take photos? Did the school find the method disruptive?; Lancaster et al., 2004). By utility, we mean how useful the method was from the perspective of youth (e.g., Was it a helpful way to share their perspectives?), community (e.g., Was information gathered worth their participation?), and adult research (e.g., Do the results from the photo-based method add additional depth to more typical interview or focus group methods?) partners.
Feasibility
From the adult research team’s perspective, this methodology was generally a feasible and useful method to conduct evaluation with youth in the context of health promotion/prevention programming, although it did have some challenges. Implementing this method within the school context was fairly straightforward; however, it is important to note some important contextual caveats. First, we implemented this method in a small, rural school where staff were highly supportive of the initiative. This was important as school staff needed to be present at both of the group meetings (due to school policy) and were instrumental in arranging youth interviews; because most youth lived far from the school and there was no public transit available, administration allowed us to conduct interviews during instructional time. The WiseGuyz program had also been in the school for several years, and facilitators had formed a positive and solid relationship with the school and with youth in the school. In a school setting with less supportive staff, it may take more creativity to implement the two group meetings and individual interviews (e.g., implementing on evenings or weekends in an out-of-school setting). Second, being in the school setting also meant youth could not take pictures on school property or that would identify somebody, constraints that were set per school policy. Though these constraints did not seem to limit the ability of youth to take meaningful photos, they may have shaped the type of photos they took (e.g., many were of nature settings). Third, being in the school setting may require careful consideration around youth payment. We had originally planned to conduct this evaluation in two school divisions. However, one of our potential school divisions had recently instituted a rule that did not allow students to be paid as part of research projects (this rule applied to research projects conducted within the school, as well as projects where students were recruited within the school setting, even if the research occurred off-site). Since WiseGuyz is a school-based program and we thus needed to recruit youth in the school setting (even if all other meetings were held off-site), this rule also applied to our project. We explained to the school division that as this was a youth participatory project, an important aspect of research ethics was paying youth photographers; as discussed by Powers and Tiffany (2006), “the work of youth researchers needs to be supported with appropriate human, financial, and logistical resources” (p. S86). However, we were not able to get an exception for our project and thus we chose to withdraw our research project from this school division. As we feel that paying youth participants is core to youth participatory ethics, researchers planning to use this method may wish to discuss with schools ahead of time the need to be able to pay youth participants. If they are not able, we would recommend that evaluators engage with youth in order to determine what is an ethical alternative to payment in that setting (e.g., creating an event that celebrates accomplishments, covering costs of food).
Acceptability
Youth photographers were asked about their experiences using the photo-based evaluation method during their individual visual storytelling interviews in order to assess acceptability. Through these interviews, it was clear that all youth photographers enjoyed the process. In particular, youth discussed that the process of taking photos for this project was easy and enjoyable.4 Youth photographers explained that early in the process, they may have forgotten and worried they wouldn’t have enough photos. However, in the progressing weeks, they found it to be a natural part of their day. As one explained, “Well I kind of forgot that first week…then my mom was like ‘oh yeah you should probably take some of those pictures.’ So then I just started kind of looking for things to take pictures of and share” (Matt). When the youth were asked what they liked and did not like about the process of taking photos for the project, they reinforced that it was fun and enjoyable. For example, one said “I found it fun, it was also good to be outside and stuff” (Gerald), and another stated:
I had no issues with it at all. Like I really enjoyed it, I think it was very, you know, a different way of expressing things. There’s like, when you have these kind of projects, most people just like write a paper on it, [or] whatever. This is a very unique way of like getting the points across. (Evan)
When students were asked to articulate what made the process easy and enjoyable, they gave examples of how they weaved photography into their routine,
It was pretty easy I think especially since it was in [photography] class as well. So you can kinda while you’re taking photos you can also think about photos you need to take for WiseGuyz and also the help from the other teachers on tips on how to take photos. (Sky)
or how they used preexisting photos as part of the project: “I just thought this was perfect, it was the one I was thinking of the second that you guys said I could take photos or use photos I’d already taken.” (John)
Youth photographers also reported feeling a sense of autonomy (i.e., independence and choice) during the project process. For example, it was important for youth photographers to take photos that were meaningful to them and to take photos at their own pace. As one youth recounted, “I really like how it was up to you to decide the story of kind of like what you felt about it. Yeah I liked how free it was to do what you wanted” (Sky). Regarding time constraints, another youth shared, “It wasn’t like there was like a super tight—there was a deadline which was set which is good. But there was also like flexibility within it so you gave us a lot of time.” (Evan)
Overall, youth photographers found the project acceptable, in particular because they were given autonomy in their work, and thus they found it enjoyable as opposed to demanding: “It was all really nice, it was actually calm because I just got to go outside for a bit and take pictures…just another excuse to go have some almost alone time. It’s very calming.” (Gerald)
Our school research partners also found the method acceptable, as documented by their active engagement in the research process and support for our project. However, as we note under Feasibility, the method may not be acceptable in all school settings, and it is important to have discussions with school partners prior to engaging in the work about what the method entails and what may need to be adapted to make the method acceptable for their setting.
Utility
From an adult research and school perspective, this method was very useful. The data we collected as part of this project were extremely rich. The use of photographs clearly scaffolded youth to articulate deep insights on their program participation, insights that may otherwise have been inaccessible (Hieftje et al., 2014). In addition, the data are very useful as part of knowledge translation and advocacy efforts. For example, time in school settings is very limited, and having data that speak to youths’ own perspective on their participation—including why the program needs to be offered in schools and during instructional time—is influential for program administrators. The dissemination of photos and stories (as opposed to stories alone) has also piqued much interest in the project from a wide variety of stakeholders (Drew et al., 2010), including community-based organizations, policy makers, schools, and academics. Per the strong utility of this method for both understanding youth experience and disseminating project findings, we will be replicating it with a group of youth involved with the justice system in the coming year. Given the focus on youth voice and active participation, we feel this method may be especially useful for use with youth whose voices are often marginalized from the public discourse.
From a youth perspective, the photos and accompanying narratives underscored the benefits of using photographs to elicit critical reflection as part of program evaluation. When the youth described their photographs, they often described them in either a literal or metaphorical manner. With regard to literal interpretation, as depicted in Figure 2A, one youth photographer expressed that his favorite picture was of the words ‘I love you' written on his arm “cuz A, my girlfriend gave it to me and B, just the picture” (Matt). However, this expression was one of few literal interpretations, as many were metaphorical. Indeed, it appeared that the photography allowed youth to be more creative and less literal in their representations. An example of how one youth generated new knowledge on what it meant to be a guy before WiseGuyz is captured in the statement:
Like, say that this is, like, the giant cloud coming over, would be like all the pressures of school and stuff. And then I’d be the little tiny hamlet over here. And then I’d be, feel overwhelmed by everything that’s happening around me and I’d be once again, more confused and stressed. (Gerald; Figure 2B)
Figure 2. Youth photographer photographs. A) A literal interpretation. Photograph #1 by Matt. B) A metaphorical interpretation. Photograph #2 by Gerald.
The camera was also described as a valuable tool for capturing youths’ experience of the program: “I like how it could, how you could just be expressive through your photos and write down whatever you wanted to and just basically talk a lot about what you think and expressing your feelings and ideas.” (Merman)
Conclusion
For researchers interested in engaging youth in participatory evaluation of health promotion/prevention programming, we feel the described method is feasible, useful, and acceptable. Overall, the main lesson learned was that this method is an effective way to get adolescent boys to think critically about what they learned in a health promotion/prevention program and produces knowledge translation documents that are very meaningful for community and youth stakeholders. Further, we believe this method makes a unique contribution because the multimodality of an image provides layered metaphors and symbols associated with youth experiences that individual interview/focus group questions alone would not have revealed with the same depth. These metaphors and symbols then allow youth to explore their thoughts and feelings about the program and articulate them in a much deeper way; indeed, all members of the research team felt that the breadth and depth of the data was much greater than achieved in the team’s collective past decade interviewing adolescents. Although the method was used here in the context of a gender-transformative program, we feel it is flexible and can be used by researchers and practitioners evaluating a wide range of health promotion/prevention programs with youth.
In regard to feasibility, our findings indicate that the school setting primarily influenced this aspect of the research. Specifically, the level of staff support—coupled with student honorarium policies—was foundational to the overall process of research implementation. Thus, prior to implementing this method in the school setting, it is important for adult researchers to explore school policy on participant payment and to gauge staff interest in supporting this type of data collection. In terms of usefulness, the rich data gathered through this method have been extremely useful to both informing research on the program and disseminating youths’ experiences in the program to the broader public. The inherent multimodality of the method also addresses its utility as a means of data collection, a mode of visual representation, and a form of knowledge translation.
This study also demonstrated the acceptability of youth participant–generated photography; youth photographers reported that they generally found the process to be highly acceptable and enjoyable. It is worth nothing that several students in our project concurrently participated in a photography class, which we were not aware of at the time of recruitment, but which may have also facilitated photo-taking. Overall, the ease of the photo-taking process appeared to facilitate critical reflection while honoring participant autonomy through a creative form of expression. This also substantiates findings of the acceptability of photo-methods of Affleck and colleagues (2012):
…the independent nature of the data collection, and the time required to take the photographs, allows the participant a greater opportunity to reflect on the photos, collect their thoughts, and formulate their discussion points without feeling rushed to do so. (p. 159)
However, while the method was generally feasible, useful, and acceptable, we also found several potential limitations to a participatory photo-based evaluation with youth, in particular the intensified level of engagement. For instance, this method requires a deeper level of involvement on the part of participants compared to other evaluation methods (e.g., surveys). It is also a more intensive method for adult researchers than typical semistructured interviews (which generally involve only one interaction with youth, and all data collection and analysis led by adults). Thus, smaller sample sizes are likely most feasible given this increased intensity. However, although only six youth participated in our initial application of this method, outcome data were robust (Exner-Cortens et al., 2020), and saturation was reached; from the research team’s perspective, the rich data were worth the sample size trade-off. While the number of youth in this study was not a limitation to gathering quality data, the limited variability in experiences was a drawback (i.e., as masculinities are dynamic and socially situated, we would ideally have five to six participants from several different contexts in future applications of this method). In preparing for our study, we noted that in Drew et al.’s (2010) work on visual storytelling, some youth had difficulties creating enough photographs and knowing what to take pictures of, in part because it required forethought and planning ahead. Several youth in our study also reported that at first they had trouble deciding what to photograph but were able to overcome these barriers (e.g., through perseverance, through using existing photographs). However, two youth ultimately decided not to continue in the project, indicating this method (as with any method) is not a fit for all youth; thus, the use of multiple methods in any program evaluation remains important. To support youth with potential barriers around remembering to take pictures, our project research assistant texted youth photographers weekly to encourage them and remind them about the project; however, we recognize that this level of engagement may be more difficult with larger samples and that the increased workload of using participatory research may pose a potential barrier for some research teams (McDonagh & Bateman, 2012). Finally, it is also important to indicate that our research benefited from working with participants who were members from an established group. Research studies working outside of this dynamic may want to consider expanding the number of group interactions to build the level of comfort among participants.
Implications for Research and Practice
This research contributes to the growing scholarship on more deeply understanding the lived experience of health promotion/prevention interventions with youth, as it is the first study (to our knowledge) that uses photography as the basis for youth participatory evaluation in the context of health promotion/prevention programming in North America. The findings from this study have significant practice implications for youth program practitioners, research evaluators, and school-based staff. Practitioners may find that there is a strong desire from many youth program participants to engage as youth practitioners and researchers. The youth in this study demonstrated a keen interest in exercising their autonomy and ability to meaningfully engage in program evaluation that will have real-world implications for themselves and their peers. The findings from this study support the emerging literature about the capacity and interest of youth to be more authentically involved in youth-based program planning and evaluation.
This photo-based study also makes a unique and important contribution to the field of participatory evaluation methods. In particular, the research design builds on the strengths of both individual and group processes: leading with interviews compliments group discussions, as the process provides students with the chance to initially explore their photographs in a one-to-one setting, with the opportunity to expand on their ideas and concepts within a larger group. This procedure also allows for different forms of participation, particularly for youth who may not feel comfortable sharing their thoughts in a larger group (or vice versa). In doing so, the method also supports individual critical reflection as well as dialogic learning and engagement.
Acknowledgments
The authors would like to thank the participating school and youth photographers. They also thank Debb Hurlock, PhD, Liam Potter, Stafford Perry, Colin Bath, and Joe Campbell for their contributions to this project. Thanks also to the Centre for Sexuality for their support of this research.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This research was supported in part by a Seed Grant from PolicyWise.
1. Gender-transformative approaches “engage men and boys to reflect critically on—and then to challenge and change—gender-inequitable attitudes and behaviors” (Brush & Miller, 2019, p. 1635). Gender-transformative approaches draw heavily on social norms theories (Brush & Miller, 2019), as well as theories that highlight the need for broader structural change (e.g., intersectional feminism; Dworkin & Barker, 2019), and are a recommended violence prevention and health promotion strategy for men and boys (Brush & Miller, 2019).
2. One potential drawback to this method is that all data were collected retrospectively. This decision was originally made for feasibility reasons; however, we found that conducting the evaluation midway through the program year allowed youth to reflect on changes in a way that may not have been possible had we used a prospective method (i.e., collecting photo-based data in a pre/post format). If a prospective design was desired, the method could be adapted to fit this format.
4. With regard to acceptability, youth were asked how they found the process of taking photos and what they liked and didn’t like about the process, and these data were then coded and themed (see “Procedures” section for analysis details). With regard to utility, a theme that emerged during the analysis process was on the types of photos that participants took (literal vs. metaphorical), which we report on here. For findings specific to program impacts, please see Exner-Cortens et al. (2020).
References
Affleck W., Glass K., Macdonald M. (2012). The limitations of language: Male participants, stoicism, and the qualitative research interview. American Journal of Men’s Health, 7(2), 155–162.
Brush L. D., Miller E. (2019). Trouble in paradigm: “Gender transformative” programming in violence prevention. Violence Against Women, 25(14), 1635–1656.
Claussen C. (2017). The WiseGuyz program: Sexual health education as a pathway to supporting changes in endorsement of traditional masculinity ideologies. The Journal of Men’s Studies, 25(2), 150–167.
Drew S. E., Duncan R. E., Sawyer S. M. (2010). Visual storytelling: A beneficial but challenging method for health research with young people. Qualitative Health Research, 20(12), 1677–1688.
Drew S., Guillemin M. (2014). From photographs to findings: Visual meaning-making and interpretive engagement in the analysis of participant-generated images. Visual Studies, 29(1), 54–67.
Dworkin S. L., Barker G. (2019). Gender-transformative approaches to engaging men in reducing gender-based violence: A response to Brush & Miller’s “Trouble in Paradigm.” Violence Against Women, 25(14), 1657–1671.
Epstein I., Stevens B., McKeever P., Baruchel S. (2006). Photo elicitation interview (PEI): Using photos to elicit children’s perspectives. International Journal of Qualitative Methods, 5(3), 1–11.
Exner-Cortens D., Hurlock D., Wright A., Carter R., Krause P. (2019). Preliminary evaluation of a gender-transformative healthy relationships program for adolescent boys. Psychology of Men and Masculinity. Advance online publication. https://doi.org/10.1037/men0000204
Exner-Cortens D., Wright A., Van Bavel M., Sitter K. C., Hurlock D., Carter R., Krause P. (2020). “To be a man is to be yourself:” Outcomes of the WiseGuyz program through photo-based evaluation [Unpublished manuscript].
Ford K., Bray L., Water T., Dickinson A., Arnott J., Carter B. (2017). Auto-driven photo elicitation interviews in research with children: Ethical and practical considerations. Comprehensive Child and Adolescent Nursing, 40(2), 111–125.
Haines-Saah R., Oliffe J. (2012). Visual methods in gender and health research. In Oliffe J. L., Greaves L. (Eds.), Designing and conducting gender, sex, and health research (pp. 127–144). SAGE.
Halsall T. G., Forneris T. (2016). Behind the scenes of youth-led community events: A participatory evaluation approach using photovoice in a Canadian First Nation community. Youth Engagement in Health Promotion, 1(2).
Hieftje K., Duncan L. R., Fiellin L. E. (2014). Novel methods to collect meaningful data from adolescents for the development of health interventions. Health Promotion Practice, 15(5), 714–722.
Kistin C., Silverstein M. (2015). Pilot studies: A critical but potentially misused component of interventional research. The Journal of the American Medical Association, 314(15), 1561–1562.
Kramer L., Schwartz P., Cheadle A., Rauzon S. (2013). Using photovoice as a participatory evaluation tool in Kaiser Permanente’s Community Health Initiative. Health Promotion Practice, 14(5), 686–694.
Lancaster G. A., Dodd S., Williamson P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312.
London J. K., Zimmerman K., Erbstein N. (2003). Youth-led research and evaluation: Tools for youth, organizational, and community development. New Directions for Evaluation, 2003(98), 33–45.
McDonagh J. E., Bateman B. (2012). ‘Nothing about us without us’: Considerations for research involving young people. Archives of Disease in Childhood—Education and Practice, 97(2), 55–60.
Oliffe J. L., Bottorff J. L. (2007). Further than the eye can see? Photo elicitation and research with men. Qualitative Health Research, 17(6), 850–858.
Powers J. L., Tiffany J. S. (2006). Engaging youth in participatory research and evaluation. Journal of Public Health Management and Practice, 12, S79–S87.
Raber M. P., Lopez K. K., Pomeroy M., Mody A., Markham C., Sharma S. V. (2016). Brighter sights: Using photovoice for a process evaluation of a food co-op style nutrition intervention. Journal of Health Disparities Research and Practice, 9(3), 2.
Rounsaville B. J., Carroll K. M., Onken L. S. (2001). A stage model of behavioral therapies research: Getting started and moving on from stage I. Clinical Psychology: Science and Practice, 8(2), 133–142.
Sitter K. C. (2018). Adapting photovoice in the classroom: Guiding students in the creation of a photovoice project. In Shapiro B. (Ed.), Actions of their own to learn: Studies in knowing, acting, and being (pp. 215–224). Brill Sense.
Skrzypiec G., Harvey-Murray R., Krieg S. (2013). The PhotoStory method as a legitimate research tool in evaluations: More than a nice story. Australasian Journal of Early Childhood, 38(3), 25.
Wang C., Burris M. A. (1997). Photovoice: Concept, methodology, and use for participatory needs assessment. Health Education & Behavior, 24(3), 369–387.
Yi-Frazier J. P., Cochrane K., Mitrovich C., Pascual M., Buscaino E., Eaton L., Panlasigui N., Clopp B., Malik F. (2015). Using Instagram as a modified application of photovoice for storytelling and sharing in adolescents with type 1 diabetes. Qualitative Health Research, 25(10), 1372–1382.
Zeldin S., Bestul L., Powers J. (2012). Youth-adult partnerships in evaluation (Y-AP/E): A resource guide for translating research into practice. ACT for Youth Center of Excellence, Cornell University.
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Deinera Exner-Cortens, Department of Psychology, University of Calgary, 2500 University Drive NW, Calgary, Alberta, Canada T2N 1N4. Email: [email protected]
Metrics and citations
Metrics
Journals metrics
This article was published in American Journal of Evaluation.
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
