Getting Your Educational Innovations “Out There”

I hear it frequently from colleagues and friends…

I’m just not a research-y person.  I see patients and teach, I don’t have time to write up articles for journals…Besides, I’m not good at that stuff…

Perhaps it is the fault of the increasing demands of our clinical work, the difficulty in recruiting academic faculty, concern about the increasing ethical and regulatory requirements associated with research and evaluation, or the blending of traditionally clinical health systems and the traditionally academic universities.  Whatever the cause, there is certainly a diminishing emphasis on “contributing to the academic conversation” among medical school clinical faculty.

In Boyer’s often-quoted model of scholarship (Boyer, 1997), the scholarship of teaching is well-defined.  It is not simply the actual teaching that is important, but the “systematic study of teaching and learning processes” that qualifies as scholarship in our current academic climate.  As physicians in an academic health center, we cannot simultaneously rail against the dismissal of teaching as an afterthought in our centers without also being ready to examine our own teaching techniques and their effectiveness in a public manner.  We do this, at the very least, through structured evaluation of our teaching.

Building evaluation into our teaching is not difficult – but does require some creativity and forethought.  I offer a conceptual model to help you organize your efforts and then some techniques and ideas that should reduce the barriers to accomplishing some evaluation…and to help get your results “out there.”

Move beyond “reaction”

An increasingly popular model of instructional evaluation is the Kirkpatrick model (Kirkpatrick, 1998).  In this model, four “levels” or perspectives of instructional evaluation are defined:

  1. Reaction – Did the learners like the instructional session?  Did they feel like they learned?  Were the instructional media (slides, notes, texts) interesting and clear?
  2. Knowledge – Did the learners improve on measures of their knowledge (tests, etc.)?
  3. Process Change – Did the learners change how they did things (in an objective, measurable way)?
  4. Results – Did the organization benefit from the training?  Originally, these theories came from business – so this was asking about business results.  In healthcare, we might ask, did the patients live longer or have less morbidity as a result of this education?

Level 4 is quite a leap for most of us thinking about how we will evaluate our CME lecture or course, but it is important for us to have in mind when designing instruction.  As a start, though, let’s just move beyond level 1.  Consider, as you are designing a new curriculum or lecture, how you might evaluate those learners at levels 2 and 3.  If you’re looking for knowledge change, can you look for sustained change?   Think of a behavior you would like to see in your learners as a result of your instruction, then think of how you would observe that behavior in what they do. Think about your instructional opportunities not simply as information delivery but as opportunities to create learning – then ask yourself, “How will I know my instruction is effective?”

Make evaluation a team sport

Partner with friends and colleagues.  Find someone with more skill than you have in whatever area you need help – they’ll often be flattered you asked.  However, ask in advance, not at the last minute.

  • If you are planning to evaluate your project and think you could use some statistics, then find someone to help with that as you are planning your project…not just when you have some data.
  • Of course, you may not need statistics – qualitative research can often serve as the basis of educational evaluation – find someone who can run focus groups or structured interviews for you and who will work with you to analyze these.
  • One of the best friends any academic faculty member can have is a medical librarian.  Remember, their actual job is to get the answer if they don’t already know it, so ask them about medical education resources, to help you with your background search for your project, or to help you find a journal that may be interested in publishing your evaluation.

Outlining and Blitzing – get your thoughts on paper

The thought of writing is often the biggest deterrent for would-be scholars.  For those starting with a idea for scholarship, it helps to write a “2-pager” – a description of a project that gets all the thoughts buzzing around your head down on paper in an organized fashion.  It brings the intangible into reality and can be shared with others as the idea is developed and refined.  But even 2 pages can be difficult…

  • Outlining can help with organizing your thoughts, getting them all out on paper and revising and refining once done.
  • Blitzing is sort of the opposite – sit down and dump all your thoughts onto the page without concern for punctuation, grammar or that “particular turn of phrase.”  Write until you don’t have any more thoughts on the subject.  Only then should you go back and revise.

Find an outlet for your publication…it’s OK to start small

We cannot all be published in Academic Medicine.  Start small.  Create a talk for your institution’s teaching academy, grand rounds or faculty development series on some aspect of your teaching.  Consider contributing to an established blog site (specialty society, academic society, etc.), to a state academy journal or a county medical society newsletter.  Try to build in space for feedback on your ideas – your colleagues can help you refine and develop your ideas.

With some forethought, organization and collaboration, you can take your teaching to the next step – to the scholarship of teaching!

 

Boyer, E. L. (1997). Scholarship reconsidered: priorities of the professoriate (1st ed., 12th pr). Princeton, NJ: Carnegie Foundation for the Advancement of Teaching [u.a.].
Kirkpatrick, D. L. (1998). Evaluating training programs: the four levels (2nd ed). San Francisco, Calif: Berrett-Koehler Publishers.