Code Review in the Classroom

Professional Software Engineers and developers regularly implement code review, i.e., they review each other’s code to identify errors, suggest improvements, and share knowledge in the team. Code reviews are useful and common in the industrial environment, e.g. they are part of the software development at Google and it is applied by more than 25K developers making more than 20K source code changes each workday [19]; however, they are not always applied as they should in academia, neither for what concerns research and development nor regarding pedagogical approaches. We consider code reviews a best practice to be applied in the academic environment.

Tools to support Code Reviews are abundant (see some of these in section 2); in this paper, we are not introducing another tool of this kind, but we propose a process on how to perform a Code Review session in the university classroom, also in the presence of large groups. Our work follows the convergent practices across several code review processes and contexts identified in [21]: it is a lightweight and flexible process; reviews happen early (before a change is committed) and quickly; code sizes are small; few people are involved; and it is a group problem solving activity. We describe how this process can help in the development of the course and how it helps students to understand colleagues’ mistakes and learn coding best practices.

In particular, students can reflect on:

• •

  Documentation: Is the code properly documented and commented? (Possible answers: It needs more work / Somewhat / Yes it is great). Where should the documentation be better?

• •

  Error handling: Does the code handle errors properly? (Possible answers: It needs more work / Somewhat / Yes it is great). Where should the error-handling be better?

• •

  Suggestions: Provide two suggestions for the author on how to improve the code.

In this work, we focus on answering the following research questions:

• •

  RQ1: What are the benefits and pitfalls for a course performing code reviews?

  Lecturers need to achieve specific learning outcomes for a course in a specific time frame. We want to explore how code review sessions can help lecturers in the process. We are also interested in exploring the consequences of adding code review sessions as part of the course. Finally, we want to determine whether there are more benefits than pitfalls: whether code reviews are worth in an academic setting.

• •

  RQ2: What are the benefits and pitfalls for students participating in code reviews?

  We want to determine whether code reviews help or hinder the learning process in students. This is important as it is directly related to students’ motivation. The more motivated students are the more engaged they are in the activity [24], resulting in taking full advantage of the activity’s benefits. There is also a need to understand what the pitfalls of the activity are so lecturers can focus on those elements.

• •

  RQ3: What aspects of the artifacts under review do the students emphasise most?

  It is important to determine these aspects so lecturers, replicating the code review activity, can pay more attention to those elements. This makes the code review more focus, affecting (positively) the outcome of the activity.

The following high-level objectives motivate these research questions:

• •

  Keep the syllabus of the course realistic: the course may not cover topics that should be covered. It may also loosely cover some of its topics, or cover some topics too extensively. We believe that conducting communication-intensive activities such as code review sessions may help uncover these problems.

• •

  Ensure good enough learning quality: we may not deliver some topics of the course well enough. The course consists of both lectures and lab sessions. While the lectures may cover up-to-date theory, the labs may insufficiently or inadequately practice that theory. Some teaching assistants may practice the topics that they appreciate more. They also may have different backgrounds; this will likely affect their perception and delivery of the material.

• •

  Ensure adequacy of the exercise itself: the setting of the code reviews may require adaptation. It was the first time we applied code reviews for teaching first-year bachelor’s. We designed the exercise based on how a code review would look like in a software company. The participants of code reviews in such companies understand very well the importance of code reviews. They also understand what aspects of the artifacts under review they should especially focus on. First-year bachelors most likely do not have that level of awareness. By studying their behaviour during code review sessions, we may learn how to adjust the setting of the exercise.

The rest of the paper is organised as follows. Section 2 gives a brief overview of the code review process in industry and academia, and shows different tools to perform reviews. Section 3 describes the process to perform code review in a classroom. The process describes both side of the activity: steps to be followed by educators and steps followed by students. Section 4 presents a case study where the process was applied to a large size course. Section 5 lists the lessons learnt after performing the code review. Finally, section 6 is devoted to conclusions.

Code inspection consists of a manual review of the code by other developers than its authors to identify its defects. Already thirty-five years ago, it was recognised as a good engineering practice, which was successfully applied in several programming projects improving productivity and product quality [7]. At that time, it was done with in-person meetings: such code inspections can be called traditional in contrast with practices that have been more recently introduced. In particular, [2] defines Modern Code Reviews as informal, tool-based and regular peer code reviews. Kemerer and Paulk [13] analyse the impact of designing and performing code reviews in assignments of a Personal Software Process course written in C or C++: they empirically verify that allowing sufficient preparation time for reviews and inspections can produce better performance; this probably explains why doing such research in a teaching environment is easier than in an industrial one, which tends to try to decrease the short-term costs. Nevertheless, such research in an industrial environment exists: as an example, Sadowski et al. [23] studies the practice of Modern code reviews at Google and its impact. Interestingly, they do not only study the impact on the codes themselves but also on knowledge transfer due to code review; indeed, at Google, knowledge transfer is part of the educational motivation for code review. This educational capability of reviews was already emphasised in [12]. Rigby and Bird [21] attempt to measure such spread knowledge across the development team through review. As recalled above, Modern Code Reviews in industry are tool-based. The pipeline followed by these tools (typically) starts by the owner of the code submitting the code to the tool. This allows reviewers to see the code and to propose changes (if needed); the tool enables both owner and reviewers to have discussions about the lines of the code. They can also propose changes on the code; finally, the code is patched or discarded. In an industrial setting, this is typically done before committing a change to the main repository. Some example of these tools are: CodeFlow [5] used by Microsoft; Gerrit [9] used by Google; ReviewBoard[20] used by LinkedIn and Mozilla; Phabricator[18] used by Facebook and Quora. However, many aspects of the academic setting do not allow existing commercial code review processes to be used and present a software solution to code reviewing in academic settings so that students can get constructive comments about the code they turn in for assignments[26]. Code review in education has also been experimented in the context of secondary school [14], where students perceived code review activities positively, and such activities helped in achieving good results in programming projects.

University lecturers who at the same time work in software production understand well the importance of code reviews, and realise the difficulties of teaching and implementing them in a class environment. Lecturers might not have a prior experience in presenting the topic, how to asses it, and even less how to coordinate the action over large groups. In this paper, we try to answer some of these questions. Some techniques applied in a university environment are described in [27]. The authors of the article discuss their approach and conclude that students actually learn from reviewing code and can assess quality, but also that generally, and understandably, do not like to grade the work of their peers. Hundhausen et. al. presents a list of best practices for implementing code reviews in an academic setting[11], the list is derived from empirical studies. Our work follows most of the practices: establishing ground rules and modelling the activity are part of the activities in phases (1) and (2) in figure 1. We did not, however, perform “mock” reviews beforehand; requiring both independent and team reviews is part of step (2) in Phase (3) (figure 1); avoiding redundancy in reviewing is achieved by the nature of our work as we suggest to code review in a more realistic scenario, students need to perform this activity before committing their code to a repository; we did not use trained moderators. Moderators were trained, but they were students of the same course, we did not take into consideration their experience.

Code review is a process in which a piece of code is manually inspected by developers other than the authors. The primary purpose of the code review is to improve the quality of software projects. Code reviews are frequent in the industry (e.g., [23]) where code is inspected before committing the changes in centralised repositories. Reviews ensure that quality code adheres to the company’s best practices. The review process in the industry happens seamlessly as part of the software development; however, freshly graduated students are not prepared to perform the review (they instead learn the process on the fly). This section describes the process of performing a code review in an academic environment.

The code review process defined in this paper assumes a project-based course that has at least six students, so at least one review session is performed. The process can handle large size courses; for instance, we applied it to a group of 129 students, a total of 22 reviews were performed (see section 4). Figure 1 depicts the whole process. The process has two main components (detailed in sections 3.2 and 3.3). The upper part of the figure shows the process that educators performing the review need to follow, and the lower part the process of the actual code review. The process for educators starts with a set-up phase in which educators need to give directions and explanations about the review session to participants. In this phase, students form groups and roles are assigned. Phase 2, preparation, is about a series of meetings that educators perform with students in the same role (e.g., meeting with owners). In each meeting, roles are further explained to students, and some skills, e.g., facilitation for moderators, are exercised. Phase 3 is the actual Code Review; see below part of figure 1. It comprises five steps. It starts with planning the review. Here, roles for each code review are introduced, goals are defined, and the code for review is distributed. In step 2, preparation, reviewers review the unit of review off-line (as suggested in [10]). There is an initial interaction between reviewers and the owner to determine whether some of the review comments can be solved off-line. Step 3 is the actual review session. All participants, reviewers, owners, and moderators meet to discuss only those review comments that could not be taken off-line. Finally, Rework, and Follow-up steps take place. During the Rework step, the owner implements the review comments that were not solved. During the Follow-up step, owners notify everyone about the changes. Once the code review is done, a Reflection phase is performed (Phase 4). In this phase, educators and participants reflect on the whole activity.

This section reports on the Code Review process performed in a large size course of a Computer Science program.

We report on the lessons learnt after implementing the code review in a classroom mentioned in previous sections. These lessons are supported by feedback provided by students via a questionnaire (that can be found in [22]). Lessons cover aspects related to the technicalities of the code review process, and some suggestions are in regards to improving the work atmosphere to avoid unnecessary difficulties and achieve productive outcomes.

Code reviews are widely used in industry. However, freshly graduated students are not prepared to undertake them. This paper describes a process to perform code reviews in an academic environment that seeks, among other things, prepare students for the future. The process defines two sub-processes to be carried out one by the educator of the course and the other one by students of the course. We applied the process to a first year, large size course in Computer Science. We also presented the findings of the activity and described the lessons learnt after the activity. This paper can be used as a guide to implement code reviews in a classroom (examples of code reviews and material can be found in [22]).

We now summarise the results of the code review activity against the research questions stated in section 1. The summary relies on the following key sources of information:

• $\star$

  Students’ evaluations of each other’s code (figure 4).

• $\star$

  Students’ responses to a postmortem reflection questionnaire that we handed out to them after the code review activity. We received 78 responses in total, which is slightly more than half of the students (all responses can be found in [22]);

• $\star$

  Our gained experience.