Wednesday, November 23, 2016


Grading Effectively and Fairly

Contributed by: Stacy Wittstock
Issues with Grading
  • Consistency is often an issue, especially when grading a lot for large classes
    • This can be individual consistency, across single tests or between students
    • It can also include consistency within teams
  • Grade inflation is also a concern, as is being overly critical in your grading
  • Grade distributions can also be a concern

Subjectivity vs. Objectivity in Grading
  • It’s impossible to exactly quantify a student’s performance over a semester
  • Also, grades subjective by nature
  • They are partly about meeting instructor expectations of how something should be written
  • They are influenced by factors that often aren’t relevant, such as handwriting legibility and ink color (Klein & Taub, 2005; Greifeneder et al., 2010)
    • They are influenced by circumstances outside the control of student and instructor
      • Comparison between students’ work
        • Contrast/order effects (Spear, 1997)
      • Past knowledge of the student’s work
        • “The halo effect,” where one’s personal impression of a student can impact how they grade that student (Malouff et al., 2013)
    • There is also a lot we don’t know about how bias impacts grading practice
      • The studies themselves may bias instructors to grade without bias (Malouff, 2008)
  • So, what’s the solution?
    • Blind grading is one possibility, though it eliminates the possibility of personalized feedback for students
      • Grading students’ work from a different section than the one you teach
      • No names/means to identify
      • Divide and conquer among TAs

Grading Exams
  • There are a variety of different exam types. For example,
    • Multiple Choice
    • True-False
    • Matching
    • Short Answer or Completion
    • Essays
  • Contrary to how they are often viewed, all of these question types can and often do exercise higher-order thinking skills, but what they don’t all do is exercise creativity and flexibility. For the first 3, you are usually locked into a given answer or set of answers, though there may be some room for discussing alternatives. Short answers and completion allow a little more flexibility, but usually, these are limited. What makes them “better” is that they’re simple to grade: the student either gave the correct answer, or they gave the wrong answer. Essays allow that creativity and flexibility, but they usually are not simple to grade, partially as a result of that same issue.

Responding to Student Writing
  • One instinct when commenting on student writing is to focus on grammar or mechanics.
    • Not only does this take a lot of time, but do students really learn much from this type of response?
  • “Minimal marking” (Haswell, 1983)
    • Point out that there are mistakes, but push students to identify the specific mistake and figure out how to correct it themselves
  • Haswell (2006) advises that we “[eschew] the traditional cover-all-bases approach to writing response and adopting a smaller task-specific, problem-specific, and learner-specific method.”
  • “Students are avid for commentary (though they may first look at the grade), but when forced to explain their teachers' comments, they misinterpret a shocking portion of it. When forced to revise, they assiduously follow the teacher's surface emendations and disregard the deeper suggestions regarding content and argumentation. They prefer global, non-directive, and positive comments but make changes mainly to surface, directive, and negative ones” (Haswell, 2006).
    • If we know all of this, what does that mean for how we grade the final product?
    • How does this problem impact the transferability of feedback from assignment to assignment, class to class?
      • These are all considerations we need to make while thinking about how to provide feedback to students

References
Greifeneder, R., Alt, A., Bottenberg, K., Seele, T., Zelt, S., & Wagener, D. (2010). On Writing Legibly Processing Fluency Systematically Biases Evaluations of Handwritten Material. Social Psychological and Personality Science1(3), 230-237. doi: 10.1177/1948550610368434

Haswell, R. (1983). Minimal Marking. College English, 45(6), 600-604. Retrieved from http://users.ipfw.edu/wellerw/minimal_marking.pdf

Haswell, Richard. (2006). The complexities of responding to student writing; or, looking for shortcuts via the road of excess. Across the Disciplines, 3. Retrieved from http://wac.colostate.edu/atd/articles/haswell2006.cfm

Klein, J. & Taub, D. (2005). The effect of variations in handwriting and print on evaluation of student essays. Assessing Writing, 10(2), 134-148. Retrieved from http://dx.doi.org/10.1016/j.asw.2005.05.002

Malouff, J. (2008). Bias in grading. College Teaching56(3), 191-192. Retrieved from http://dx.doi.org/10.3200/CTCH.56.3.191-192

Malouff, J. M., Emmerton, A. J., & Schutte, N. S. (2013). The risk of a halo bias as a reason to keep students anonymous during grading. Teaching of Psychology, 40(3), 233-237. doi: 10.1177/0098628313487425

Spear, M. (1997). The influence of contrast effects upon teachers’ marks. Educational Research39(2), 229-233. Retrieved from http://dx.doi.org/10.1080/0013188970390209

Friday, November 18, 2016

PLANNING LESSONS -- BACKWARD DESIGN

Contributed by: Mengyuan Xiao

CONCEPTS (WHAT):
-- Backward Design: “An approach to creating effective, learner-centered lesson plans that is the reverse of how many instructors traditionally plan to teach.[1]
-- So to understand backward design, first we need to get familiar with a “lesson plan.”
Lesson Plan: “Describes the learning process in enough detail that any instructor could see the flow of the class.[1]
Usually a lesson plan includes these six elements[1]:
• Learning outcomes for the lesson
• Materials and equipment
• Introduction (review/preview/attention grabber)
• Outline of learning activities and assessments
• Wrap-up/Summary
• Pre-class preparation/Homework
-- Backward Design is the reverse to Traditional Lesson Plan. So what is a traditional lesson plan, or how will you design a lesson plan intuitively?  In the discussion, most people showed that they would intuitively start a lesson plan by considering what knowledge or activities the instructors want to teach or conduct.  In contrast, the backward design starts with the lesson outcomes and follows this order:
Desired Learning Outcomes -> Assessments -> Activities

STEPS (HOW): 
-- Step 1: Articulate learning outcome(s) - What should students know and be able to do?  (e.g. Students will be able to prove if an algorithm is correct.)  Learning outcomes had better be concrete, measurable and achievable by the end of a specific lesson.
-- Step 2: Assessments - What evidence will demonstrate that students are making progress towards the learning outcomes?  (e.g. Students will be able to complete an exam with word problems and short answer.)  A good way to design the assessments is to incorporate practical problem that students will encounter when they start to apply the knowledge outside of the class.  It is also important to make the feedback of assessments available to students and enable them to evaluate their class study by themselves.
-- Step 3: Activities – What exercises will help students develop the skills and knowledge needed to meet your learning outcomes?  (e.g. Students will work in teams to solve practice problems.)  Assessments and activities are sometimes interchangeable because some assessments may act as activities that prepare students for the final course assessment.
-- Step 4: Check alignment – How do the assessments and activities help students achieve the learning outcome(s)?  Learning outcomes, assessment and activities should align between each step. Pay attention to integrate these three elements.

BENEFITS (WHY):
Through discussion, we found out several benefits of backward design:
-- In the class, instructors may be distracted by students’ questions, etc. Lesson plans with backward design help instructors to focus on the lesson outcomes and the learning goals.
-- Sometimes, students get the feeling that the content in the exams are not taught in the class. Utilizing backward design makes instructors to think more critically about the reflection of learning goals through assessment and the consistent design of teaching activities.
-- Clear desired learning outcomes could be used to clearly communicate expectations to students. Students may refer to these learning outcomes to prepare for exams.

OTHER COMMENTS:
-- In addition to lesson plan, backward design could be applied to many other activities involving learning and information communication.
-- Q: Is backward design distinguishable from the traditional model when designing lesson plans?    
-- A: Yes, especially in syllabus design.

REFERENCE:
[1] The TA’s Guide to Effective Teaching at UC Davis. (cee.ucdavis.edu/docs/TA_Guide_2015-Accessible.pdf)


Wednesday, November 2, 2016

PLANNING FOR DIVERSE LEARNERS

Contributed by: Rachel Anderson & Chelsi Florence

So far this quarter, we’ve discussed various means of inclusion in terms of diversity in the classroom (last week) and learning styles. This week, we looked at another form of inclusion in terms of classroom curriculum and activity through an approach known as “Universal Design for Learning.”

Universal Design for Learning
Video for your viewing pleasure to briefly introduce UDL: https://www.youtube.com/watch?v=bDvKnY0g6e4

Diversity is finally (or at least on paper) becoming more of an institutionalized part of the agenda in parts of higher education. And we as educators must be recognize and act on the fact that a classroom should not be a space where certain students are excluded because the curriculum (institution and instructor) favors a certain kind of student over others. This is where (we hope) UDL can come in.

What is UDL?
“Universal Design for Learning (UDL) applies to the design and creation of learning experiences, processes, and spaces that are accessible to all students and not just a select few, thereby making learning Equitable” (source: “Dig Deeper: Universal Design for Learning (UDL)”).

Learning made equitable both in terms of student access and success. And all of this is done ideally while still maintaining the integrity of the course and achieving its objectives.

Learning happens in three parts. We as educators aim to facilitate learning by helping as many students as possible gain knowledge, skill, and enthusiasm.
    • Knowledge is the WHAT (content)
    • Skills  are the HOW (to think and do)
    • Enthusiasm is the WHY (the “so what?” we often ask)

So, the question we ask is how does this actually happen?
Part of this requires flexibility. There are three aspects of flexibility we can engage with:
  • Multiple methods of (re)presentation
  • Multiple options for participation
  • Multiple means of engagement

The first refers to presentation of the content (the knowledge) to students. The second is various ways to participate or engage with knowledge. The last is means to engage or express what they have learned. This isn’t necessarily new or groundbreaking--we often deal with these using different terminology. For example:
  • (Re)presentation  = Lecture
  • Participation = Section or other in-class group activities, etc.
  • Engagement = Exams, papers, and other assessments

What does matter is this approach is calling for intentional and critical reflection and response on the part of the instructor to craft curriculum and activities that would allow access to all students. It is asking for flexibility on the part of the instructor first and foremost to facilitate access to learning for students.

Why should we consider UDL in approaching our curriculum?
Well, there are a few reasons for this.
  • This built-in flexibility provides into a wider range of options for students to choose from meaning the curriculum adapts to the student, rather than the other way around.
    • Did you get that? It properly places responsibility on the us to attempt to first make the curriculum accessible to student
  • When people believe choice is given, people are more likely to engage in the activity (Lewin, 1952)
  • Choice increases intrinsic motivation, effort, task performance, perceived competence, and preference for challenge (Patall, Cooper, & Civey Rovinson, 2008)

Questions to reflect on:
  • Is there anything problematic with this? Are there assumptions being made or not covered by this approach?
  • Can we truly make things “equitable” (fair/impartial)? Does this actually make the classroom a fair, equal opportunity space?
  • What are potential barriers to implementing this?
    • Research - (Katz)

Additional Resources

References
Katz, J. (2015). Implementing the Three Block Model of Universal Design for Learning: effects on teachers' self-efficacy, stress, and job satisfaction in inclusive classrooms K-12.International Journal Of Inclusive Education 19(1). http://www.tandfonline.com/doi/abs/10.1080/13603116.2014.881569

Lewin, K. (1952). Group decision and social change. In G. E. Swanson, T. M. Newcomb, & E. L. Hartley (Eds.),Readings in social psychology(459-473). New York: Holt.

Patall, E. A., Cooper, H., & Civey Robinson, J. (2008). The effects of choice on intrinsic motivation and related outcomes: A meta-analysis of research findings. Psychological Bulletin, 134 (2), 270-300. Doi : 10.1037/0033-2909.134.2.270.



Some classroom challenge scenarios we discussed in pairs and then as a group, and how we might address them with the principles of UDL:

You're teaching a large introductory lecture class in your department, and you're concerned that students are not paying attention during lecture.  You see them using their phones or sleeping.
We agreed that the problems here were a lack of engagement and participation.  We suggested using a diversity of styles, including moving the room or participants around, using clickers or otherwise involving the class, and potentially developing a new phone policy.

You are the TA for a discussion section of 20 students.  The class requires students to read and discuss the technical literature.  There are international students in the classroom who seem to struggle with the vocabulary and are hesitant to speak up in class.
We came up with a few solutions to encourage participation including providing a list of terms to everyone, creating group work that may encourage student confidence, and giving students a class to turn in written assignments if they prefer those to oral communication.

You're the instructor for a lab class which requires hands on work.  You have a blind student in the class, and you're not sure how to incorporate her into your lesson planning.
One of us had previously taught a blind student in a statistics lab, and said that the most effective strategy to benefit all learners was to clearly present and read all information aloud (not to rely on written communication), and to be flexible in grading assignments.

You have some students who require extra time on tests.
We discussed how this might be an opportunity to think about how to allow all learners to better demonstrate their skills and knowledge, rather than creating an exception for students with a letter from the SDA.  Perhaps giving everyone a take-home test that could be completed in any length of time would benefit both those “on the margins” as well as others who would like more comfort while test-taking.

INCLUSIVE ENVIRONMENTS FOR LEARNING

Diversity as an Asset in the Classroom
Contributed by: Mari Rodriguez

The second week of fall quarter presentations in the Graduate Teaching Community is geared toward using diversity as an asset in the classroom. The objectives of the presentation are for participants to familiarize themselves with the backgrounds of UC Davis undergraduates, define inclusive classroom environment, and consider strategies to create inclusive classrooms. To achieve these goals, I started the session off with a warm-up activity. I asked participants to complete an identity map, using their hobbies, interests, characteristics, and goals (and anything else they could think of!) to fill in the blank spaces. This activity led to a discussion on how we define ourselves, as well as the recognition that one’s identity is an extremely complex and limitless concept; just as we bring varying expectations and experiences to our meetings, undergraduate students enter the classroom with a whole range of identity characteristics. Who are UC Davis undergraduate students and how can we leverage their different backgrounds to establish an inclusive and diverse classroom?

UC Davis undergraduate students are diverse not only with regard to their age, race, and ethnicity, but also in terms of their experience to learning. For example, 26% of UC Davis undergraduate students in 2015 were transfer students, 42% were first-generation students, and 10% were international students (UC Davis Undergraduate Admissions and UC Info Center, Fall 2015). While similar data on faculty members is not available, there are significant gaps with respect to racial and ethnic representation between faculty members and undergraduate students. Since UC Davis outlines several goals for creating a diverse population of scholars by 2020, including Hispanic Serving Institute aspirations, it is crucial that we, as TAs, establish welcoming and inclusive learning environments.

Inclusive classrooms are “classrooms in which instructors and students work together to create and sustain an environment in which everyone feels safe, supported, and encouraged to express his or her views and concerns” (Saunders & Kardia 2016). But how is this environment achieved? The GTC discussed a variety of ways in which we can create welcoming learning environments, including allowing space to declare preferred pronouns, offering multiple modes of learning, and accommodating students with familial responsibilities. To reinforce these concepts, I gave participants a slip of paper with either a classroom bias or a solution to overcome the classroom bias. I asked participants to find their “match” and discuss how they might improve the proposed solution. The GTC did a great job of recognizing why the classroom biases were potentially harmful to students and brainstorming how they could improve the suggested solutions!

Work Cited:


Saunders, Shari and Diana Kardia. 2016. “Creating inclusive college classrooms.” University of Michigan Center for Research on Learning and Teaching. Available at http://www.crlt.umich.edu/gsis/p3_1.