A Virtual Programming Lab for Moodle with automatic

assessment and anti-plagiarism features

Juan C. Rodríguez-del-Pino, Enrique Rubio-Royo, Zenón J. Hernández-Figueroa
Departament of Informática y Sistemas
University of Las Palmas de Gran Canaria
35017 Las Palmas de Gran Canaria, Spain

Abstract - This paper describes VPL, a Virtual

Programming Lab module for Moodle, developed at the
University of Las Palmas of Gran Canaria (ULPGC) and
released for free uses under GNU/GPL license. For the
students, it is a simple development environment with
auto evaluation capabilities. For the instructors, it is a
students' work management system, with features to
facilitate the preparation of assignments, manage the
submissions, check for plagiarism, and do assessments
with the aid of powerful and flexible assessment tools
based on program testing, all of that being independent
of the programming language used for the assignments
and taken into account critical security issues.
Figure 1. VPL homepage
Keywords: programming learning, virtual lab, automatic
VPL is designed to achieve the following goals:
assessment, plagiarism.
 To be an open source tool, freely distributable and
1 Introduction capable to be enriched with external contributions.
For this reason it is distributed under GNU/GPL
Achievement of computer programming skills requires license.
a lot of training by means of real program-development
assignments. Managing and assess the students'  To be independent of the programming language.
submissions for those assignments could be a very To use a particular programming language it is
complex task. Availability of tools to organize the only required that the appropriate compiler is
assignments, receive and storage the submissions, support installed in the jail system (see section II for
automatic or semi-automatic assessment and provide details). Currently, there are available jail systems
feedback could be very helpful. Furthermore, integration with installations for Ada, C, C++, C#,
of that kind of tools into a Learning Management System FORTRAN, Haskell, java, Octave, Pascal, Perl,
is an essential feature in order to improve their PHP, Prolog, Python, Ruby, Scheme, SQL and
performance. VHDL. Any user can install other languages if
required.
Some tools for this purpose have developed along the
time. STYLE [1] and CAP [2] focus on the automatic  To provide a very simple development
evaluation of program's style and syntax. PASS [3] focus environment in order to smooth the learning curve
on assessment capabilities. Some tools focus on execution to the beginners.
for a specific programming language, as PACER [4], for  To provide tools to support automatic and semi-
ELI, a language similar to C, or HoGG [5] and ELP [6] automatic assessment, including tools for
for Java. More general tools focus on course management plagiarism checking.
features, as Ceilidh [7], CourseMaster [8] and Boss [9].
 To be conscious of security issues. To avoid
This paper describes VPL, a Virtual Programming Lab security breaches when executing students' code,
module for Moodle, developed at the University of Las all executions are performed in a separated and
Palmas of Gran Canaria (ULPGC) and released for free restricted environment (the above mentioned jail
uses under GNU/GPL license. Downloading of VPL is system).
available at http://vpl.dis.ulpgc.es; also an on-line demo
site is available at http://demovpl.dis.ulpgc.es. Figure 1 Following sections describe the VPL architecture, the
shows the VPL homepage. types of activities that can be done using VPL, how to
configure those activities, the use of VPL to assess To provide execution in console the Moodle servers
submissions an check for plagiarism, and our experience need to open at least two ports, a larger number is
as VPL users along the test phase of the module. recommended. It takes more time than usual to execute
the PHP scripts that run a submission so it is necessary
increase time limit in the PHP configuration.
2 VPL architecture
1.2 Network topologies
1.1 General features
The VPL module uses a double proxy to
VPL is composed of three elements: a Moodle´s communicate, by a side with the Internet clients, to attend
module, a browser-based code editor and a jail component their requests, and by other side with the jail servers, to
(Fig. 2). perform the running tasks associated to those requests.
The code editor is a java applet providing basic This permits a variety of network topologies. The simplest
features to edit, run, debug and evaluate programs code in topology runs both the jail server and the Moodle server in
a very simple code-development environment. To use full the same computer, although they have to communicate
features a web browser with JavaScript and support for via an intranet. This solution loses the security advantages
Java 1.5 applets is required. provided by the isolation of the servers on different
computers. A more suitable topology joins a Moodle
The Moodle's module provides the typical features of server with one or more separate jail servers, which may
this kind of component (backup and restore, integration be in a private network.
with the grading book, course reset, events control, role-
based access …) but also specific features such as: A more powerful topology, that improves the
submission management, assessment support and anti- resources spending, is to share multiple jail servers among
plagiarism features. The VPL module requires a Moodle multiple Moodle servers (Fig. 4). This configuration may
version 1.9.x and PHP5 or higher. adapt itself to workload peaks by changing the number of
jail servers in use by a Moodle server in order to attend in
The jail component is the server in charge of compile a proper way the variations on the requirements of the
and executes the code submitted by the students in a execution tasks. The drawback of this configuration is that
secure environment. It runs a linux chroot command to the jail servers must be in a public domain in order to
provide a restricted version of the host file system with make them available to all the Moodle's servers without
some read-only limitations. To run or evaluate a increasing the network complexity.
submission is required to have at least one jail server. The
jail service needs Ubuntu (recommended) or Red Hat
compatible linux distribution.. XMLRPC +
raw TCP

VPL Jail 1/1

http/https + XMLRPC +
raw TCP
console I/O (raw TCP)

Moodle VPL Jail 1/k

Server 1
Browser +
Java plugin

VPL Jail S1/1

rs
e
rv

VPL Jail S1/1

Figure 2. VPL components
se
ed
ar
Sh

The jail server attends requests for both interactive and VPL Jail S2/1
VPL Jail S2/2
non-interactive executions (Fig. 3), the difference between XMLRPC +

then is that the second type requires to the request data raw TCP

include a key, a server and a communication port, which http/https +

console I/O (raw TCP)
VPL Jail 2/1

are used to redirect the execution input/output data. Moodle XMLRPC +

Browser + Server 2 raw TCP
Java plugin
VPL Jail 2/q

Figure 4. A complex net topology

Steps of a non-interactive running request Using multiple jails serves not only support scalability
and improves performance, but also provides fault
tolerance. When an execution request is received by the
VPL module, it takes the list of available jail servers and
randomly selects one that is not marked as having a
previous fail into a specific range of time. Then VPL
sends the server an availability request; if the response to
Steps of an interactive running request this request is true, the execution request is assigned to the
server, else a new server is selected. If no server is found,
Figure 3. Types of execution request the process is repeated taken into account the servers
previously failed.
3 Types of VPL activities network, or do not permit the editor's copy and paste
features.
VPL can be used to configure, manage and assess a
As for our-classroom activities, multiple attempts of
range of learning activities which can be classified by type
solution could be tried. In both cases, the student can get
or scope [10]. By type, the learning activities may be:
immediate feedback for each try, if so configured.
examples, cloze or puzzle exercises, and code
development exercises. By scope they may be: out-
classroom tasks, or in-classroom exams. 4 Configuring VPL activities
1.3 Learning activities by type 1.5 Basic configuration form
Examples are activities where the students are The creation of a VPL activity begins by filling its
provided with both the description of a problem and the basic configuration form, which includes, as for majority
program code that solves that problem. The students may of Moodle's modules: activity's name, short description,
interact with the code (running or debugging) to see how availability period (with visibility and due dates), grading
it works. options, grouping... In addition, an VPL activity may
Examples may be mutable or immutable, depending include data as: maximum number of files to submit,
on if the student can or no modify the code. An immutable maximum size of those files, restrictions on editing,
example must be marked as "example" in the activity's network, password... (Fig. 5).
configuration window. The student who modifies the code
of a mutable example can always reset it to its original
state.
Cloze exercises and puzzle exercises are especially
appropriate for beginners. In the same way than the
previous activity, they provide the students with the
description of a problem and the program code that solves
that problem, but now the student must modify the code.
In the case of the cloze exercises, portions of the code Figure 5. VPL basic configuration form restrictions
have been deleted and the student has to fill in the blanks
in order to do the code works in the appropriate way. When the basic configuration form is completed, the
instructor can configure five other groups of features: full
In the case of the puzzle exercises, the code has been description, tests cases, options, requested files and
disordered and the student has to sort it in order to do the advanced features.
code works in the appropriate way.
A code development exercise provides the students 1.6 Full description and test cases
only with the description of a problem (although some The Full description tabsheet shows a html editor
code may be sometimes included). The student has to where the description of the problem to solve must be
develop the code to solve the problem using the written to be shown to the student.
appropriate techniques. This kind of exercises is a
traditional way to achieve programming skills at any level The Tests cases tabsheet permits configure simple
by means of an intensive training in solving problems, in input-output based test cases to check programs
the believe that: 'doing programs is how we can learn to correctness in order to elaborate an assessment reports
programming'. both to provide feedback to the students and to support
grading. The configuration of each test case must include:
1.4 Learning activities by scope case short description, input to test the program, expected
output for that input, and grade reduction when the test
Attending to the conditions under what a VPL's case fails (Fig. 6).
activity can be done; there are two ways to use VPL: out-
classroom tasks, or in-classroom exams.
Out-classroom tasks are designed as long-term
activities to be done into a time period that may extend by
days or weeks. During the active period of the activity the
student may try so many solutions as he or she likes.
Out-classroom activities can be done anywhere,
without direct supervision of an instructor, although help
Figure 6. Example of test case configuration
can be obtained from the instructors as long as they offer
it in some way (e-mail, forums, face-to-face...).
1.7 The Options tabsheet
In-classroom exams are activities designed to be done
in a short-term and in a restricted environment, under The Options tabsheet (Fig 7) serves to configure some
instructor's supervision. So, they can be configured to general options for execution of submissions tests. The
require a password, be done in a specific local area first feature in the Options tabsheet serves to indicate if
the activity is based on another VPL activity, on this way execution, debugging and evaluation (running the tests) of
it is possible to build a hierarchy of activities which share the submissions.
some features via heritage.

Figure 7. Options tabsheet

The rest of the options in the Options tabsheet are for

configuring if the students can run, debug or evaluate their
submissions, if the exercises will be evaluated just when
submitted, and if the results of the automatic evaluation Figure 9. Execution files for advanced testing
will become in the final assessment or they will be used
only as a help for a human evaluator. The Advanced tabsheet includes also the configuration
of other features (Fig. 10) which usually do not need
1.8 Requested files modification, such as limits for execution (time, memory
The Requested files tabsheet (Fig. 8) serves to put and disk use) or designation of specified jails for the
obligatory names for the files that the student have to activity.
submit to complete the activity. The max number of files
was put in the basic configuration form. The studen can
submit any number of files, until the max, with any Figure 10. Advanced options tabsheet
names, but if names for files are specified in the
Requested files tabsheet, the student must submit at least
those files. 5 Assessment support
VPL offers both automatic and computer-aided
assessment. Selection of one of them is made by checking
the appropriate option in the Options tabsheet, as
described above.
The key for the assessment support is the
configuration of the program tests; this can be made by
simply listing a set of input-output tests cases or by
configuring a more complex tests framework.
Figure 8. Requested files tabsheet VPL automatically executes the configured tests and
produces a report that includes a list of failed tests, with
The instructor may provide initial contents for the explanatory comments, and a grade proposal. This report
specified files, by example, to configure a cloze exercise. could be used in three ways: to provide feedback to the
students while they are developing their solutions to the
1.9 The advanced tabsheet exercise (formative assessment), to produce the final
The configuration options described in the previous assessment if automatic evaluation is configured, or to
sections suffice to configure an activity with an useful test help a human evaluator to assess the submission.
system. The Advanced tabsheet serves, among other
things, to configure a more powerful test system.
The basic tests are black-box tests based on console
input-output that check for functionality of the submitted
code. Using the advanced options we can configure more
powerful functionality tests, based on unit testing
frameworks than can be similar, for example, to the well-
known JUnit fo Java [11]. In addition to the unit tests, we
can also configure other types of test like style checking or
coverage tests, so producing an assessment report that
takes into account not only functionality, but many other
parameters linked to the code quality. To configure the
advanced tests we can include some files (Fig. 9) to be
joined with the submitted files to prepare the tests. We Figure 11. Evaluation window
must also configure some script files that will control the
 When evaluating a submission the human evaluator Affected, NA = Not Affected, SA = Slightly affected,
uses a window (Fig. 11) where appears the assessment ASC = Affected by Size of Changes, ANC = Affected by
report produced by the test system and a list of the Number of Changes).
comments added to other submissions previously
evaluated. The human evaluator can modify the report TABLE I. METRICS AFFECTATION
deleting comments, adding new comments or reusing

Complex change
comments from the list, and recalculate the grade. Change

Code reorder
Comments

Systematic
Identifiers
Name of

change
6 Plagiarism checking
Plagiarism is a real problem ([12], [13], [14]) that must Metric
be faced from different perspectives: formation, Metric 1 F F NA SA A
prevention, and prosecution. Both, prevention and Metric 2 F F NA ASC ASC
prosecution can benefit from technological development Metric 3
as plagiarism itself does. F F A ANC ANC

VPL includes a tool to check plagiarism among source

code. The main goal of this tool is to detect plagiarism Experience using the tool has shown that some
among submissions for a task in a course, but it can plagiarism cases present not a one-to-one relationship, but
include other sources, like submissions for the same task a group relationship where usually nobody has a detailed
in previous semesters, or similar tasks from other courses, knowledge of all the participants, for example, student A
which are a probable source of plagiarism. may lend its work to students B and C, without mutually
knowledge of B and C; student C may lend the work to
The process to find similarities among source files is student D, without knowledge of A and B, and so on [16].
composed of three steps: tokenization, comparison and To provide the reviewers with information about that kind
clusterization (Fig. 12). of event, the system incorporates algorithms to identify
clusters of most similar files.
The visualization system of the anti-plagiarism tool
permits to visualize lists of pairs of similar files, clusters
of similar files, or file-to-file similarities (Fig. 14).

Figure 12. Steps to find similarities among source files

Tokenization is the process to get a normalized

signature from every file in order to perform an efficient
comparing to find similarities among them. It is
composed of three phases: lexical analysis, filtering and
normalization (Fig. 13). The lexical analysis extracts the
tokens that represent the elements of a program (they
depend on the programming language), then those tokens
are filtered to delete those that are irrelevant for
comparison, and finally expressions are normalized into a
canonical form, producing the program signature. Figure 14. File-to-file visual comparison

7 Experience of use
VPL current version was released in September 2011,
and is used for nine courses in the current academic year
(2011/2012) with a total of 1181 students. At the moment
Figure 13. Tokenization phases of writing this paper (the second semester is not ended)
376 VPL activities have been configured for those
Signatures are normalized representation for the courses, producing 18664 submissions (these counts only
source code files, extracted from them in order to optimize the final submissions, not the multiple tentative
the comparison process. The form of the signature submissions done during the submission period).
depends on the metric to be used in comparison. VPL use
A previous release was tested during the academic
three different metrics [15] which, when comparing two
year 2009/2010 using only a course with 208 students and
signatures, produce a number in the range 0.0 to 1.0,
42 activities, producing 1315 final submissions, and then
where 0.0 means "totally equals" and 1.0 means "totally
it was used in the academic year 2010/2011 for five
different". Using three metrics takes advantage of the fact
courses, with 661 students and 264 activities, producing
that they are affected in a different way by the
5140 final submissions.
modifications of the code. Table I shows the effect of
typical code changes on the metrics (F = Filtered, A =
 During this period, VPL was publicly available and Las Palmas de Gran Canaria to the development of VPL.
was developed and used for about 50 academic We also wish to acknowledge the coordinators and
institutions over the world. The comments and teachers of the courses that have used VPL during its test
suggestions made by these users also have contributed to phase for their useful comments.
the current version.
Some polls made to the students during the academic References
year 2011/2012 reveal that VPL activities were the best [1] Rees, Michael J. «Automatic assessment aids for Pascal
valued learning resources, with about a 80% of students programs.» SIGPLAN Not. (ACM) 17 (1982): pp 33-42.
declaring that the utility of VPL activities for their [2] Schorsch, Tom. «CAP: an automated selfassessment tool to check
learning was very high and about a 16% declaring that it Pascal programs for syntax, logic and style errors.» SIGCSE '95:
was high. Proceedings of the twenty-sixth SIGCSE technical symposium on
Computer science education. ACM, 1995, pp 168-172.
[3] Choy, M., S. Lam, CK Poon, FL Wang, YT Yu, and L. Yuen.
8 Conclusions and future work «Towards Blended Learning of Computer Programming
Supported by an Automated System.» Blended Learning, 2007: 9.
This paper describes VPL, a powerful tool to manage [4] Palakal, Mathew J., Frederick W. Myers, and Carla L. Boyd. «An
and assess computer programming exercises freely interactive learning environment for breadth-first computing
distributable under GNU/GPL license. The main science curriculum.» SIGCSE Bull. (ACM) 30 (1998), pp 1-5.
advantage of this tool is its integration in a popular [5] Morris, D.S. «Automatic grading of student's programming
learning management system, as Moodle is. This assignments: an interactive process and suite of programs.»
integration provides access to all the features of that kind Frontiers in Education, 2003. FIE 2003. 33rd Annual. 2003. S3F-
1-6 vol.3.
of platforms.
[6] Truong, Nghi, Peter Bancroft, y Paul Roe. «A web based
A major feature of VPL is its capacity to produce environment for learning to program.» Australian Computer
complete assessment reports based on program testing. Society, Inc., 2003, pp 255-264.
Moreover the required program tests can be configured in [7] Foubister, S.P., GJ Michaelson, and N. Tomes. «Automatic
assessment of elementary Standard ML programs using Ceilidh.»
a very flexible way, ranging from simple input-output Journal of Computer Assisted Learning 13 (1997), pp 99-108.
tests to complex combinations of unit tests, coverage tests
[8] Higgins, Colin A., Geoffrey Gray, Pavlos Symeonidis, and
or style tests. Athanasios Tsintsifas. «Automated assessment and experiences of
teaching programming.» J. Educ. Resour. Comput. (ACM) 5
Another important feature of VPL is the embedded (2005): 5.
tool to check submissions for plagiarism. It is important
[9] Joy, Mike, Nathan Griffiths, y Russell Boyatt. «The boss online
because plagiarism is a big problem in academia, as many submission and assessment system.» J. Educ. Resour. Comput.
studies have shown. (ACM) 5 (2005): 2.
[10] Rodríguez-del-Pino, J.C.; Rubio-Royo, E.; Hernández-Figueroa,
The current version of VPL (1.4) runs under Moodle Z. Uses of VPL. 5th International Technology, Education and
1.9.x. By the beginning of the 2012 summer will be Development Conference (INTED). 2011, pp 743-748.
released the version adapted to Moodle 2.x, including new [11] Beck, K. and Gamma, E. JUnit cookbook Available on-line at:
and improved features. http://JUnit.sourceforge. net/doc/cookbook/cookbook.htm. 2002
The architecture of VPL will be re-engineered to [12] Crittenden, V. L.; Hanna, R. C. & Peterson, R. A. The cheating
culture: A global societal phenomenon. Business Horizons, 2009,
include the case when the Moodle server is running in a 52, 337-346
cluster. The problem with the clusters is that, using the [13] Hughes, J. & McCabe, D. Academic misconduct within higher
current design, the communication between the console education in Canada. Canadian Journal of Higher Education,
and the jail for an interactive execution must be done 2006, 36, 1
through the Moodle server, but, if there are a cluster of [14] McCabe, D. L. Cheating among college and university students: A
servers responding the requests, it is not possible know North American perspective. International Journal for Educational
what server has to manage the communication. Integrity, 2005, 1, 1-11.
[15] Rodríguez-del-Pino, J.C.; Rubio-Royo, E.; Hernández-Figueroa,
The Java applet editor will be changed by one Z. Fighting plagiarism: metrics and methods to measure and find
developed using HTML5. Currently the Java applets do similarities among source code of computer programs in VPL. 3rd
not work properly on systems like iPad, or tablets running International Conference on Education and New Learning
Technologies (EDULEARN). 2011, pp 4339-4346.
Android.
[16] Lesner, B., Brixtel, R., Bazin, C., & Bagan, G. (2010). A novel
framework to detect source code plagiarism: now, students have to
Acknowledges work for real!. Proceedings of the 2010 ACM Symposium on
Applied Computing, ACM, 2010, pp 57-58.
We wish to acknowledge the support of the
Department of Informática y Sistemas of the University of