Designing Intelligent Review Forms for Peer Assessment: A Data-Driven Approach

This evidence-based practice paper employs a data-driven, explainable, and scalable approach to the development and application of an online peer review system in computer science and engineering courses. Crowd-sourced grading through peer review is an effective evaluation methodology that 1) allows the use of meaningful assignments in large or online classes (e.g. assignments other than true/false, multiple choice, or short answer), 2) fosters learning and critical thinking in a student evaluating another’s work, and 3) provides a defendable and non-biased score through the wisdom of the crowd. Although peer review is widely utilized, to the authors’ best knowledge, the form and associated grading process have never been subjected to data-driven analysis and design. We present a novel, iterative approach by first gathering the most appropriate review form questions through intelligent data mining of past student reviews. During this process, key words and ideas are gathered for positive and negative sentiment dictionaries, a flag word dictionary, and a negate word dictionary. Next, we revise our grading algorithm using simulations and perturbation to determine robustness (measured by standard deviation within a section). Using the dictionaries, we leverage sentiment gathered from review comments as a quality assurance mechanism to generate a crowd comment “grade”. This grade supplements the weighted average of other review form sections. The result of this semi-automated, innovative process is a peer assessment package (intelligently-designed review form and robust grading algorithm leveraging crowd sentiment) based on actual student work that can be used by an educator to confidently assign and grade meaningful open-ended assignments in any size class.

Designing Intelligent Review Forms for Peer Assessment: A Data-Driven Approach
Z Beasley, L Piegl, P Rosen
ASEE Annual Conference & Exposition, 2019

Ten challenges in CAD cyber education

The advancement of technology and its application to the field of education has caused many to re-examine the merits and pitfalls of cyberlearning environments. Though there is a wealth of research both for and against its mainstream use, there is a consensus that much work remains to be done in key areas such as collaboration, course content, personal learning environments, and engagement. CAD and cyberlearning share a common goal: to communicate information effectively. Unfortunately, many aspects well understood in CAD have been overlooked in online education. In this paper, ten key challenges and their implications for CAD cyber education are discussed. The purpose of this paper is not to provide a dismal outlook for cyberlearning, but to incite discussion, research, and development into these areas with the anticipation of a viable and attractive alternative to traditional classroom education.

Ten challenges in CAD cyber education
ZJ Beasley, LA Piegl, P Rosen
Computer-Aided Design and Applications 15 (3), 432-442

Leveraging Peer Review in Visualization Education: A Proposal for a New Model

In visualization education, both science and humanities , the literature is often divided into two parts: the design aspect and the analysis of the visualization. However, we find limited discussion on how to motivate and engage visualization students in the classroom. In the field of Writing Studies, researchers develop tools and frameworks for student peer review of writing. Based on the literature review from the field of Writing Studies, this paper proposes a new framework to implement visualization peer review in the classroom to engage today’s students. This framework can be customized for incremental and double-blind review to inspire students and reinforce critical thinking about visualization.

Leveraging Peer Review in Visualization Education: A Proposal for a New Model
A. Friedman, P. Rosen
IEEE 2017 Pedagogy of Data Visualization Workshop