Master Mentor Curriculum

Appendix B

Program Evaluation

The Master Mentor Program
Evaluation

Cynthia B. Roy, Ph.D.

“The object of teaching a child is to enable him to get along without his teacher”
(Elbert Hubbard, 1865-1915).

Introduction

I took this quote from a fourth semester performance report of a Master Mentor to her fellow participants and instructors. She was noting that her mentees were sending questions directly to her for “the answer,” rather than to the online group she established. “I am looking at Perry’s Main Line of Development, levels 1-3, a dependency on the ONE in authority to have all the answers, and am taking action from the start to steer my mentees to the upper levels of development.”

This awareness and understanding of where learners need to be is indicative of the success of the Master Mentor program. Working to encourage new learners towards independent learning as she was encouraged indicates that the basic tenet behind the MM program has now become a guiding philosophy for these master mentors. The first cohort has discovered how to mentor adult learners so that these learners become capable of self-assessment and self-improvement. I cannot think of a better indication of the success of the Master Mentor program. This program has engaged adult learners in learning as discovery and in transforming complex information in ways that connect with their previous experiences (Conceição-Runice and Daley 1998). The MM program has brought together experienced professionals (both learners, faculty and expert consultants) to share their expertise, to learn from organized knowledge bases, and to collaborate in pooling their experience, knowledge and thinking to prepare effective assessment, training, and feedback strategies for interpreters in their area.

“It is interesting to see them (the second Cohort) wrestling with the same issues we did; I feel for them and at the same time, can see the tremendous value we’ve gotten, individually and collectively, from our own explorations in this course. I think it’s interesting that none of us has run over there and given any so-called “answers,” clear evidence (to use a Dennis term) that we all have come a long way, I say!” JD, Cohort I. (emphasis mine).

The four-course content scaffolds engagement with intellectual content and learners’ practical experience into meaningful and authentic tasks. The learners actively construct knowledge through discussion and creative, thoughtful preparation of assignments such as the three portfolios. These learners are engaged in not only the construction of their own knowledge and resources but are fellow collaborators in the work of fellow learners (research has demonstrated, in medical school teaching research, for example, that students learn more from each other than they do from interactions with instructors, reading textbooks and listening to lectures). Authentic partnerships with a team of interpreter educators in online presentations and discussions, chat rooms to practice the conversational skills required of a mentor, use of portfolios to demonstrate competence and expertise, and a mentoring project are the hallmarks of this outstanding program.

Online learning

Conducting this program online has enabled working professionals from across the US to take the course, earn graduate credit, study mentoring in an academic climate, and profit from the expertise, experience and knowledge of both instructors and other learners. One of the online instructors who has been an active participant from the beginning, Robert Lee, noted that the participants built an online community through their range of knowledge, information and comfort with technology. He observed a feedback loop: the participants had to begin an assignment, think about it, do it, get feedback from others about their work, and then do it again, think about it again and return for more feedback. Participants experienced discussing both the process and product; working online gave them the luxury of time to think and work rather than focusing on the instant responses a classroom requires. “They focused on the work rather than chatting.” And online work continues in one way or another; it’s continual, not stopping just because a classroom hour is up. In a classroom, a facilitator may have to wrap up a discussion in twenty minutes, but the same discussion online can continue not only for a few days but also throughout the course as students revisit questions and concerns.

The use of chat rooms to practice talking to each other as mentor and mentee was valuable as well. Participants used the opportunity to put into practice ways of talking with each other before they accepted mentees. Ways of talking that promote not only improvement of skills but also that promote a wider, deeper understanding of the practice of an interpreter.

“I was able for the first time [during the fourth semester] to actually put to use and try out some of the skills I learned in the MM program. It felt so wonderful to be able to put theory into practice and confirm that these mentoring tools learned in the program actually work!” DJ, Cohort I.

Online facilitators/expert instructors:

Dr. Winston is an acknowledged expert in educating interpreters and interpreter educators and in the development of web-based courses for academic credit. Her knowledge of curriculum development and philosophy of facilitating online learning is current with the innovative thinking in providing graduate coursework via the Web. She provides commentaries for each topic heading, pointing out pivotal quotes, making connections to interpreting, asking thoughtful questions, and making available the widest possible range of resources for learning. Her comments to the entire group as well as the 1,000 emails she sent and answered during a single semester (Spring 2000) demonstrate her commitment to facilitate an environment where students actively construct knowledge and apply experience to develop abilities to wrestle with complex ideas and theories, organize it in comprehensible ways, and then use it to create mentoring projects that produce better trained and more skilled interpreters for a lifetime.

Dr. Winston has created a learning environment of exposure to a variety of and contact with many expert interpreter educators who have advanced degrees, long experience in the field, experience with online learning, and who have contributed significant personal feedback and correspondence. Dennis Cokely is the expert consultant for the portfolio project and was an involved participant during first course discussing the portfolio project when it first began and again in person during the summer. Robert Lee, Christine Monikowski, and others were available for discussions about the theory and practice of mentoring. These experts monitored discussions, talked to students, and encouraged exploration of ideas. All of the expert consultants who have participated are experienced with adult learners and ways to foster self-learning, and pose thoughtful questions to guide learning. This collaborative teamwork was coordinated throughout by Dr. Winston.

Materials:

The texts, readings, and CD presentations by interpreter educators are valuable resources and allow participants to have access to a variety of theoretical models. Many of the online facilitators either developed models of interpreting themselves or have extensive experience with a model. Using the new multicultural curriculum and its authors provided students with the emerging concepts and ideas for talking to and communicating with diverse individuals. The plethora of experts in interpreting and mentoring assured learners of exposure to all the various theories of interpreting, assessment, instruction, and feedback techniques. The CD Rom presentations were presented in multiple formats (signed and written) for viewing and re-viewing. Instructors were available for online questions and discussions, and thus, for example, each participant could ask and discuss questions and concerns about the portfolio projects with its lead facilitator, Dr. Cokely. His thoughtful comments and discussion avoided offering answers but rather offered alternative ways of understanding. Other facilitators/instructors such as Robert Lee, Dr. Christine Monikowski and Sandra Gish were present online and during the two-week summer meeting to present or clarify their perspectives, their approaches to improving interpreting skills, and to offer insights on mentoring. Assessment tools such as the Educational Interpreting Performance Assessment (EIPA) and research by Dr. Marty Taylor were presented by their authors and made available for learners to use.

Courses:

The four-course sequence was designed and developed through the participation of many experts and reflect what they believe is essential in educating interpreter mentors. The content reflects current approaches and new developments not only in learning, interpreting, assessment, and other areas but also in the emerging study of mentoring. The courses cover expected content, beginning with the first course, which is introductory in nature surveying approaches to mentoring and introducing the portfolio project. For example, in many disciplines, and particularly in writing, portfolios have come into wide use as a way of demonstrating competency and expertise over a range of topics and issues. As editor of Innovative Practices in Teaching Sign Language Interpreters (2000), I invited a chapter explaining the use of portfolios to demonstrate student interpreter readiness to graduate. Portfolios are currently in wide use demonstrating the range of a person’s expertise and are used to hire educators and other professionals.

The second semester covered language assessment, feedback and mentoring, business practices, and spent time focusing on how to work with different people in ways that help them learn and grow, working on ways to mentor at a distance, and what is effective at a distance, while revisiting issues from the first course and continuing the work on portfolios.

The third semester was held over the summer and provided a two-week onsite visit to the University of Colorado. Students really enjoyed meeting each other face-to-face as well as the instructors they’d gotten to know. This course continued with more focused practice and application of the knowledge, philosophies, and approaches to mentoring. Learners continued to learn and develop strategies for both assessing interpreting skills and providing strategies for making interpreting skills better. They continued to practice mentoring each other while assessing their own growth and skills as mentors.

One student who was unable to attend the onsite participated in an online version of the course. The online version was designed to integrate the work of both the onsite and online students. The format was successful, although the single student online did not have as much interaction with the others once the two-week onsite was finished. To supplement this, he chose to use his vacation to visit Boulder, and work face-to-face with Dr. Winston to review the materials and activities. He was also provided with the opportunity for online and face-to-face interaction with classmates. One classmate was able to work face-to-face, allowing practice and hands-on learning. The face-to-face component of the online section was not part of the original course design. It was created and implemented in response to the needs of this student. However, in retrospect, this model of combined online with face-to-face work was effective and should be further developed for future years. As more graduates of the program are available to provide face-to-face interactions around the country, the implementation of this model will become more feasible.

The Portfolio projects:

This four-semester project was designed to help participants discover the value of using portfolios in a mentoring relationship. All too often mentors in the field simply tell mentees what they are doing wrong at a specific moment in time. Portfolios allow mentors to gather evidence of what the mentee knows and can do in a variety of situations over time. The MMs have learned the value of portfolios in gathering information and evidence, in assessment and feedback, and most importantly assisting others in learning to self-assess and self-correct. The project was conceived with few guidelines so that participants could raise questions and issues and to allow them to tap into their own experience and creativity to raise the possibilities for inclusion in their portfolio. While, at the time, creating the portfolios led to much anxiety, confusion and frustration, Dr. Cokely, the expert consultant for this project noted:

The level of anxiety over the details seems to be running rather high. Please know at the outset that I will supply precious few details and directions. The resolution to the issues that have been raised (and that will be raised) reside within each of you and within the discussion that this intentional ambiguity is intended to create.

To each of these questions I believe that the answer is the same: “how long is a piece of string?” This answer is not intended to be smug or demonstrate a level of superiority; rather it is because each of us must search within our experience for what makes sense to us; then (and most importantly for this online course) we must risk sharing our constructs with others in order to gauge the extent to which it makes sense to others.

Risky? Yes, but isn’t that what learning and forging new frontiers is all about?

Even while in the midst of frustration, learners admitted that the portfolio was one of the best learning tools despite being hard.

Learners:

The learners who applied for and were chosen for the MM program are exceptional. They came for a variety of reasons: graduate credit, informally mentoring for a long time, could not find suitable academic studies for mentoring, and a need to learn about mentoring as their home states are beginning to require mentoring for beginning interpreters. Most students had adequate academic preparation although some had not completed a bachelor’s degree. Most learners were older and female, had years of experience as interpreters, and many were also interpreter educators or informal mentors to inexperienced interpreters in their area. Several had master’s degrees or were working on them. They were clearly highly motivated, energetic, intelligent, and overwhelmed. On the whole, they’ve posed interesting and thoughtful questions to each other and produced superior work.

Issues and Concerns:

Students complained during the summer about their frustration, lack of understanding the goals and reasons, and the heavy workload. A web-based environment requires participants to manage their time, process information, plan and manage their resources, and evaluate their own work. For example, during the first course postings grew overwhelming in number, but also during the first course Dr. Winston set up a forum on the bulletin board for discussions unrelated to the specific reading or assignment. Students took initiative and lessened the amount of postings, replied privately, and managed themselves. As they told me, “We caused a lot of our own pain.”

Other problems and solutions are ongoing. For example, with the new cohort, Dr. Winston has encouraged them to keep a log of the actual time they spend on assignments and online discussion in order to begin a discussion about ways in which time can be spent effectively. The new cohort has been separated into smaller groups and the number of postings to be summarized for the required summary discussions has been reduced. Dr. Winston and her team of facilitators rely on learner input and confer frequently in order to assess assignments, student discussions, and other content and management matters.

Other concerns revolved around the logistics of the final project. Several learners have projects within school systems and the exact when, where, and how were in constant flux as well as the necessary permission forms. However, these issues have largely been taken care of by the learners within their individual states and individual school district requirements. Forms have been developed and produced by these experienced professionals some of whom were already working in these settings, in different states, with different requirements.

One of the most interesting turns has been Dr. Winston’s decision to invite Cohort 1 to greet and be available to Cohort 2. Here are some examples of the messages from Cohort 1 to Cohort 2:

“You don’t need to spend a lot of time stressing out about making all of your work, postings, etc. “perfect” in light of the instructions. Instead, do work that is meaningful to you, be ready to justify it in light of the general criteria of any given assignment (like the portfolio) and you will be fine. The course is not about just jumping through the hoops—the goal is that you find information that is relevant, meaningful and helpful to you. I think we spent a lot of time stressing about assignments instead of doing what we thought was right for us and justifying it in a thoughtful fashion.” PG, Cohort I.

“And to echo Patty’s advice—take all the wonderful things it (the program) has to offer, but don’t stress too much on the details. Be creative—make choices and be able to back them up.” MM, Cohort I.

“It is great looking back to see all of our frustration and secretly realizing it was almost all for naught!” LE, Cohort I.

When teaching is not instructor-driven and learning is self-directed, the ultimate focus is on the learner, and most learners are not accustomed to learning in this manner. Forming one’s own meanings by actively integrating previous experiences with new understandings and concepts is hard, especially if you’ve never been asked to learn this way. Thus, it seems natural that frustrations arose and immersion in doing assignments often obscured the goals and reasons. However, solutions were arrived at, assignments proved exciting and intriguing, and awareness emerged. Most important, these learners learned how to learn, how to create meaningful ways to mentor others, and how to justify their creations.

Results:

There has been significant learning evolving into practice, mentoring of superior quality, which will impact both beginning, and working interpreters in multiple ways, and practical applications of working with adult learners in ways that encourage self-assessment and self-development skill work.

One of the most difficult things about improving someone’s skill is that the person has to be ready to change, ready to learn. The Master Mentors know this and have learned strategies for talking with others about their work; this theme is throughout the course and can be seen in each of the quotes I’ve used.

“It is amazing, when I think of all we 12 have learned over the past three semesters and now all of you will learn, how fortunate our fields of interpreting, deaf education, and interpreting education are to have a course like the Master Mentor course available to teach us more of what we thought we knew so much about.” MM, Cohort I, to Cohort II.

I have completed this report from extensive readings of course postings in all four courses to date, and from in-depth interviews with students, faculty, expert consultants, and grant and university participants. It has been especially noteworthy to see the progression of student postings from the first semester until now, as they learn and grow. A final review of this pilot program, after Cohort I has finished, as well as periodically afterwards, would add to the evaluation of this program, as will the ongoing evaluations that are routinely conducted for the new cohorts at the end of each semester.

Evaluator expertise:

I hold the Ph.D. in Sociolingustics from Georgetown University and have been involved in educating interpreters since 1979. My book Interpreting as a Discourse Process was published by Oxford University Press and I am the editor of Innovative Practices for Teaching Sign Language Interpreters published by Gallaudet University Press. With Robert E. Johnson, I designed and wrote the core curriculum for the Master’s degree in Sign Language Interpreting at Gallaudet University and was a faculty member there for several years. I am currently an associate professor and director of a bachelor’s degree in ASL/English Interpreting at Indiana University in Indianapolis. I am on the advisory board for the TIEM project and have five semesters of experience teaching online for the Professional Development certificate program. I bring to this evaluation content expertise in interpreting, writing and linguistics, and experience teaching interpreting, interpreting educators, and teaching online.