Sukhjinder Singh
Sukhjinder Singh
Abstract—A sandbox is a system that provides a safe environ- or network. The efficacy of static analysis has been greatly
ment for analysis and deployment of malware samples while all reduced over the years due to the proliferation of metamorphic
the activities of the malware are captured and logged. DRAKVUF malware with malware authors using complex techniques such
is an open-source black box analysis sandbox based on the
popular Cuckoo sandbox system. The current implementation as anti-analysis to perform detection evasion [4].
of DRAKVUF has shortcomings in terms of the web user Basic dynamic analysis and advanced dynamic analysis also
interface, which restricts user customisation of the analysis have drawbacks; namely, the number of samples that need
environment variables. This project will extend DRAKVUF’s to be analysed by malware analysts. This puts a burden on
web user interface to support new capabilities such as, an hardware resources used to analyse the samples [4]. Modern
operating system with various configurations and easy selection
and assignment of environment variables that are currently malware can also leverage anti-analysis techniques for dy-
hidden within the DRAKVUF configuration files or command namic analysis. This is done by the detection of monitoring
line interface. The queuing system of DRAKVUF is also extended environments and hiding in unmonitored corners of the system
to allow for multiple samples to be analysed at once. The [4].
reporting component is significantly improved by implementing Sandboxes are an advanced dynamic analysis method for
the capability to generate and send comprehensive reports to a
given email address provided by the user during submission. The running untrusted programs in a safe environment without
reporting interface also provides historical data on previously fear of harming “real” systems. Sandboxes comprise of vir-
submitted malware samples. tualised environments that often simulate operating systems
The improved DRAKVUF system is hosted on an ECS server and network services in some fashion to ensure that the
and is accessible to staff and students to submit and analyse software or malware being tested will function normally [2].
potentially malicious samples. The success of this project is
determined through a successful deployment and fully opera-
However, they do not offer the opportunity to customise how
tional malware sandbox of DRAKVUF sandbox instance and a the malware samples should be evaluated. This leads to results
developed web interface. being imposed due to restrictions on the choice of operating
systems, installed packages, how long the execution will last,
and many other things.
I. I NTRODUCTION
II. T HE P ROBLEM
M ALWARE analysis is the process of examining a piece
of malicious software (malware sample) to understand
its behaviour and purpose. The information provided by this
Dynamic malware analysis requires a safe and isolated envi-
ronment to run malware samples. This can be done in a variety
analysis can be used to develop defences against malware, of ways, sandboxes being a very safe way of execution and
such as antivirus signatures and firewalls [1]. analysis [3]. However, sandboxes can be resource intensive,
There are two types of malware analysis: “static” and detectable, and complex to set up and operate [4].
“dynamic”. Static malware analysis is a technique that ex- DRAKVUF is an open-source software that provides a
amines the code and properties of a malicious file without foundation for a malware analysis sandbox [5]. It provides a
executing it. Dynamic malware analysis is any examination platform for stealthy malware analysis as its footprint is nearly
performed after executing malware [2]. This generally requires undetectable from the malware’s perspective [5]. DRAKVUF
the malware sample to be executed in a system that is set provides a basic web interface that gives users limited cus-
up in a closed, isolated virtual environment so that it can be tomisability on how the sandbox will run the malware samples
studied thoroughly without the risk to a production system. and how reports will be delivered to users.
Malware analysed using the static analysis technique is much The current web interface for the DRAKVUF sandbox lacks
safer than using the dynamic analysis technique. Basic static essential features and functionalities, limiting its usability and
analysis and advanced static analysis are the two steps of hindering efficient malware analysis. Namely the interface
the static analysis method used to analyse malware [3]. On does not support the selection of multiple virtual environments,
the other hand, basic dynamic malware analysis involves it restricts the duration of malware sample execution, and it
running malware samples in controlled environments, such does not have a job management system, which will need
as virtual machines, to observe their behaviour and identify to be implemented for managing submitted jobs. This defi-
malicious activities. Advanced dynamic malware analysis goes ciency inhibits effective management of DRAKVUF instances,
beyond basic observation by utilising techniques like code and impedes analysis of submitted jobs, and fails to provide a
memory analysis, network traffic monitoring, and behaviour comprehensive solution for malware analysis.
profiling to gain deeper insights into the malware’s capabil- The aim of this project is to extend the capabilities of
ities, evasion techniques, and potential impact on a system the current DRAKVUF web interface by allowing users to
customise their analysis environment and provide an easy-to-
This project was supervised by Masood Mansoori and Lisa Patterson use interface for managing malware sample submission and
ENGR 489 (ENGINEERING PROJECT) 2023 2
queuing. When the malware sample is launched and evaluated, an avenue to observe malware execution by allowing external
there will be functionality added to DRAKVUF to email the access to the state of the virtualised hardware components.
results to the user. This technique is commonly referred to as Virtual Machine
Introspection (VMI) [4]. To access the components externally
III. P ROJECT O BJECTIVES DRAKVUF uses active VMI through breakpoint injection to
A successful implementation and solution of DRAKVUF hijack an arbitrary process within the VM to initiate the start
will meet these functional requirements: of the malware sample [4]. Breakpoint injection is a technique
used to inject a breakpoint into a running process. When this
• Be remotely accessible through a web interface.
is done, the process will stop executing at the breakpoint
• The system must show the available virtual machines and
and DRAKVUF will be able to modify it and continue the
operating system and their respective configurations for
execution. By using existing processes running within the VM,
the user to select.
DRAKVUF does not introduce any new code or artifact into
• The system must allow management of multiple instances
the analysis VM, thus greatly improving stealthiness [4]. In
of DRAKVUF, including shutting down or restarting the
contrast, the Cuckoo sandbox injects a Python script into the
instances, and managing the submitted jobs.
analysis VM which can be detected by malware.
• Support multiple operating systems and versions such as
DRAKVUF can also track changes and deletions to files
Windows 7 and Windows 10.
within the VM [6]. This is done using a technique called
• The system must allow configuration attributes of
shadow paging to track the changes made to files. Shadow
DRAKVUF to be altered and set through the web inter-
paging creates a copy of each file in the sandbox and the
face such as: which VM to execute malware on, analysis
copy is called the shadow page [6]. DRAKVUF can read
time, modules included in analysis.
the contents of the shadow pages and compare them with the
• The system must allow submission of a file to be anal-
original files to see what changes were made to the files. This
ysed by the operating systems above, through the web
is very helpful for malware analysts to determine how different
interface.
types of malware work and help identify and track malware
• The system should allow a user to download the reports
infections.
generated by DRAKVUF or view them through the web
interface.
• The system must allow a report to be emailed to a
V. R ELATED W ORK
designated email address provided by the user. There are a number of open-source sandboxes available to
And the following non-functional requirements: analyse malware samples. Many popular or defacto malware
• Efficiency: Concurrency of analysing many malware
sandboxes have been analysed to weigh their pros and cons
samples is to be maximised while performance overhead compared to DRAKVUF sandbox.
for analysing a single sample is to be minimised.
• Stealth: The monitored environment should not be able A. Cuckoo
to detect the presence of DRAKVUF. Cuckoo is a sandbox technology that is widely used by
• Isolation: DRAKVUF should be isolated from the anal- researchers and malware analysts in the industry to carry
ysis virtual machines to protect against tampering. out dynamic malware analysis. Cuckoo accomplishes this by
creating a virtual environment like DRAKVUF, however, to
IV. BACKGROUND R ESEARCH analyse the malware, an in-guest (agent) module is required.
There are several pieces of literature outlining the capabil- Due to the metamorphic nature of many modern malwares,
ities of the DRAKVUF sandbox from reputable sources such they have the potential to hide themselves or act benignly
as Intel [4] and the Institute for System Programming of the in the analysis environment if the agent is detected [7].
Russian Academy of Sciences [6]. Both discuss the challenges DRAKVUF solves this problem by utilising the breakpoint
that DRAKVUF solves around the anti-analysis techniques injection technique to execute the dynamic malware analysis
that malware employs to hide from malware analysis. [7].
DRAKVUF uses the Xen hypervisor to get around the The architecture diagram of Cuckoo is shown in Fig. 1. It
challenges that modern malwares present with anti-analysis consists of three main components, the cuckoo host machine,
techniques [4]. The Xen hypervisor offers novel techniques the virtual network, and the analysis guests. There is also
to eliminate blind-spots created by kernel-mode rootkits by the analysis manager and agent modules that are passed into
extending the scope of monitoring to include kernel internal the analysis guests. The user interacts with Cuckoo from
functions and to monitor file system accesses through the ker- the host machine either through a command-line interface
nel’s heap allocations. It also enables DRAKVUF to improve or through a web interface like DRAKVUF. The analysis
stealth by starting the execution of malware samples without manager is responsible for submitting malware samples and
leaving traces in the analysis machine [4]. This is a feature that collecting the reports after the malware analysis is completed
other popular sandbox solutions like Cuckoo and CWSandbox [7]. The Cuckoo host interacts with the analysis virtual
do not possess. machines though the virtual network and an in-VM Python
DRAKVUF is implemented using the virtualisation technol- agent module is installed within the guest OS to start the
ogy on Intel Central Processing Units (CPUs). This provides analysis automatically when the malware sample is received.
ENGR 489 (ENGINEERING PROJECT) 2023 3
The Python agent is responsible for receiving the malware Cuckoo consumed more disk space than DRAKVUF when
sample, executing it, and sending the analysis report back to memory dumping was enabled [7]. However, DRAKVUF used
the analysis manager [7]. more disk space when memory dumping was disabled.
B. CWSandbox
CWSandbox was one of the first dynamic malware analysis
systems to utilise a sandbox environment for monitoring the
interaction between the OS and the malware [4]. It operates
similarly to Cuckoo where it injects a kernel driver into
Windows OS that hooks all exported APIs to intercept the
Fig. 1: Cuckoo Architecture Diagram [7] system call performed by user-space programs. This is not
as advanced as DRAKVUF as it is vulnerable to detection
Cuckoo and DRAKVUF were compared side by side in and tampering and the interface provided by the sandbox is
[7], and the superior sandbox solution is determined through insufficient for tracking kernel-mode rootkits [4].
a series of experiments. The criteria for determining which
C. Sandbox Solution Comparison
sandbox solution was best was:
1. Duration and throughput analysis A summary of comparisons is provided in Table IV:
2. Space analysis Criteria / DRAKVUF
Cuckoo Sandbox CWSandbox
The first experiment that was conducted compared the Parameter Sandbox
duration of time that it took for Cuckoo and DRAKVUF - It does
to analyse malware samples and the throughput of each X - It uses an not use any in- X - It uses an
sandbox. Throughtput determines how many malware samples in-guest agent, the guest agent, in-guest agent, the
Stealth-
malware may not the malware malware may not
can be analysed by a system in a given amount of time. The iness
exhibit its true exhibits its exhibit its true
experiment used a set of 40, 100 and 200 malware samples. malicious nature. true malicious malicious nature.
These tests illustrated that DRAKVUF took less time than nature.
Cuckoo to analyse them and that DRAKVUF also had a - It can trace - It can trace - It can trace
higher throughput than Cuckoo [7]. Based on the experiment System all system calls all system calls all system calls
call initiated by initiated by initiated by
the authors concluded that DRAKVUF is a faster and more malware. malware. malware.
efficient tool for dynamic malware analysis than Cuckoo
X - It cannot - It can trace X - It cannot
(results shown in the Tables I and II). Kernel
trace out kernel all kernel trace out kernel
function
functions. functions. functions.
40 Samples 100 Samples 200 Samples
DRAKVUF 22 min 34 s 50 min 1 h 41 min 39 s - It supports - It supports X - It requires
Cuckoo 32 min 16 s 1 h 5 min 23 s 2 h 8 min 53 s Autom- the submission of the submission of manual interaction
ation malware samples malware samples to upload and
TABLE I: Duration to carry out malware analysis [7] automatically. automatically. analyse files.
- Windows XP/ - Windows XP/
Guest OS 7/8/10, 7/8/10,
Throughput = Total Number of samples analysed success- support Linux 2.6
- Windows 7
Linux 2.6
fully / Duration (Time taken) or above or above
Workstation Player, this made the choice for which hypervisor Name Home Desktop
to go with very easy. CPU Intel Core i7-10700KF
RAM 32GB RAM
GPU NVIDIA RTX 3050
TABLE V: DRAKVUF Hardware and OS Requirements TABLE IX: Dell Optiplex 7000 Specifications
university’s network when we had to remotely deploy it to the The project inherited ReactJS, a widely acclaimed JavaScript
network as defined in the project requirements. library known for enabling the development of highly re-
sponsive and dynamic user interfaces. While the choice of
D. Operating System (Host and Guests) ReactJS was not a selection per se, its inherent benefits,
such as its component-based structure, efficiency, and robust
The choice of operating systems for both the host and guest
ecosystem, were instrumental in continuing the development
machines is a crucial aspect of the design, directly impact-
process. The existing codebase, built upon ReactJS, facilitated
ing the efficiency, security, and overall performance of the
a seamless continuation of the enhancement and refinement
malware analysis environment. In this context, we evaluated
of the DRAKVUF web UI, ensuring an interactive and user-
various operating systems, considering their compatibility with
friendly interface for efficient malware analysis and data
the hardware, support for Xen hypervisor, security features,
visualisation.
and ease of use.
We explored a few different operating systems for the
host known for their robustness, security and support for F. DRAKVUF Architecture
virtualisation. The architecture diagram is illustrated in Fig. 2, based on
1) Windows 10: Windows 10 was considered for its user- the DRAKVUF System, the core of the diagram is a detailed
friendly interface, robust security features, and widespread use. look at the DRAKVUF system, showcasing different compo-
However, the need for a UNIX system to execute specific nents and their interconnections, represented by rectangles and
commands and scripts efficiently, and the compatibility with arrows to indicate data and process flow.
Xen hypervisor, were factors that influenced our decision.
2) Ubuntu 18.04 LTS: Known for its stability and security,
this OS was another strong candidate. However, we were
inclined towards Ubuntu 20.04 LTS due to its enhanced
features, improved security, and extended support, ensuring a
future-proof and reliable host environment.
• Malware Samples: Points to the types of files being and deny them delivery to their email servers. The reports
analysed. also vary in size and this could lead to issues when trying
• LibVMI: A library for virtual machine introspection. to attach them to emails. Providing the user with a link to
• System Calls, Analysis, VM, Introspector, User Space, the report would ensure that the emails would reach their
VMI Memory Acquisition, Kernel Space, Kernel Func- recipients without ending up undelivered or in their junk/spam
tions: These labels provide insights into various processes folders.
and areas within the virtual memory and malware analysis There were also considerations made to what tool was going
ecosystem. to be used to send the emails. The following were the options
• Virtualised Memory, Xen, Hypervisor, Hardware: Indi- that we considered:
cates the underlying infrastructure supporting the virtual • EmailJS
memory system and malware analysis. • SendGrid
• ‘mail’ command on Linux
G. Submitted Sample Lifecycle EmailJS facilitates the sending of emails directly from
The lifecycle diagram is illustrated in Fig. 3, for the submit- JavaScript (frontend), eliminating the need for server-side
ted samples on DRAKVUF’s web UI. This outlines what the code. It supports various email services, is easy to setup, and
DRAKVUF system does once a sample is submitted. It helped offered a user-friendly API.
with setting up the email feature where the report would be SendGrid is a cloud-based email delivery service that assists
retrieved after finishing the analysis. We can see when and businesses in sending transactional and marketing emails. It
where the sample analysis results are stored. is known for its scalability, reliability, and comprehensive
The lifecycle of the malware analysis consists of five main analytics.
parts as seen in Fig. 3: The mail command on Linux is a utility that allows users to
send emails from the command line. It is straightforward but
1. User job submission.
may require additional configuration and lacks the flexibility
2. Job dispatch to one of the VMs.
and features provided by dedicated email services.
3. Malware analysis.
After a thorough evaluation, we chose EmailJS. EmailJS
4. Raw result sent back to host machine.
offers a straightforward integration process, making it easy to
5. Postprocessing of results in readable format.
embed the email sending functionality within the DRAKVUF
web UI. It eliminates the need for server-side code to send
emails, reducing complexity and enhancing the security of the
application. It also supports a wide range of email services and
allows for the customisation of emails templates, ensuring that
users receive well-formatted, informative emails.
The integration of EmailJS into the DRAKVUF web UI
involves utilising its API to trigger emails upon the completion
of malware analysis. Users who provide their email addresses
during the submission of samples for analysis will receive
Fig. 3: Submitted Sample Lifecycle notifications with a link after the analysis is completed. The
email templates are designed to be informative yet concise,
ensuring users receive relevant data in an easily digestible
H. Reporting Emails format.
One of the identified project requirements was to implement Within EmailJS there are options to setup an email address
a feature in the current web UI that would send an email that the reports are sent from. We created a Gmail account,
to a provided email address containing the finished report [email protected], which is a placeholder for an
from DRAKVUF. This functionality would ensure that users official Victoria University email address. The setup was
are promptly informed, allowing for timely review and action straightforward, it required signing into the Gmail account
based on the analysis results. We explored several methods through a Google API integration within EmailJS. This was
to implement this feature, each evaluated based on reliability, the only step that was required for EmailJS to send emails as
ease of integration, and user experience. the Owhiti Sandbox email. The EmailJS tool uses Google’s
There were two options that we came up with for what the mailservers to send the reporting emails.
email would contain. These were:
• Email the entire report as an attachment VII. I MPLEMENTATION
• Email a link to the generated report In the subsequent sections of this chapter, DRAKVUF’s
Each approach had pros and cons but in the end we chose to implementation process is discussed, including the configura-
provide a link to the generated report rather than providing an tion of DRAKVUF and DRAKVUF Sandbox. The following
attachment. This was due to the fact that malicious samples sections provide an overview of the tools used during the
would be submitted for analysis, email providers could flag development phase, the deployment process, and the technical
hashes and key details of the email attachment as malicious aspects of the implementation.
ENGR 489 (ENGINEERING PROJECT) 2023 8
installed and placed in a folder (/opt/procdot/procmon2dot) a link to the analysis results, is sent to the user. The email
where DRAKVUF could access and use it. sending process is handled by EmailJS, ensuring reliability
After the installation was completed using the ‘draksetup and security.
postinstall’ command provided by the DRAKVUF sandbox sendEmail(link) {
stack, we could navigate to the web UI (http://localhost:6300/) if(this.recipientEmailAddress){
and upload some samples to ensure the analysis engine was link = webLink + link;
const templateParams = {
configured successfully. to_email:
this.recipientEmailAddress,
C. Web UI Modification Environment message: link
}
To make modifications to DRAKVUF’s existing web UI a
working DRAKVUF Sandbox instance was required. From the emailjs.send(serviceID,
templateID, templateParams,
deployment setup previously, we had downloaded the source publicKey)
code for the DRAKVUF Sandbox. The code needed to be
linked with the current sandbox deployment, the following .then((res) => {
console.log(`res: ', res);
commands were used to do this: this.clearRecipientEmailAddress();
• cd drakcore/drakcore/frontend })
.catch((err) => {
• npm install console.log(`err: ', err);
• export REACT APP API SERVER=http://localhost:6300/ });
• npm start }
}
After these prerequisites were completed we could begin
with the Web UI modifications with a NodeJS development The implementation was thoroughly tested to ensure the
server that would demonstrate live changes to the web UI on reliability and accuracy of the email notification. Each email
the address, http://localhost:3000/ contains a direct link to the specific analysis results on the
1) Reporting Emails: The implementation of the reporting DRAKVUF web UI, ensuring users can easily access and
emails feature was a critical step in enhancing the user review the findings.
experience of the DRAKVUF web UI. This feature ensures 2) Analysis Time: The implementation of an analysis time
that users are promptly notified via email with the results of slider was a significant feature for the web UI. It provides
the malware analysis. users with the ability to specify the analysis time when
The EmailJS library was imported into the project, enabling uploading a sample for evaluation. This feature ensures that
the direct sending of emails from the client-side application users have flexibility and control over the duration of the
without the need for a backend server to handle email sending. analysis, allowing for a more customised and user-centric
The ‘emailjs-com’ library was integrated into the application experience. Malware also does not finish execution within a
as shown below. set timeframe, having this slider makes it easier for the user
import emailjs from `emailjs-com'; to control how long the analysis is carried out for.
A dedicated ’MinuteSlider’ component was created in React
The service ID, template ID and public key were configured to facilitate the user’s input for specifying the analysis time.
to authenticate and enable the email sending process through This component is rendered as a slider input element, allowing
EmailJS. These configurations were essential to ensure the users to easily adjust the analysis time according to their needs.
security and integrity of the email sending process.
class MinuteSlider extends Component {
const serviceID = `service_------'; render() {
const templateID = `template_------'; return (
const publicKey = `--------------'; <input
onInput={this.props.onInput}
Functions were implemented to set, get, and clear the className=``custom-range"
recipient’s email address. These functions ensure that the email type=``range"
address is correctly handled, stored temporarily for sending the name={this.props.name}
min={60 * this.props.min}
email, and then cleared to maintain privacy and security. max={60 * this.props.max}
setRecipientEmail(recipientEmailAddress){ step={60}
this.recipientEmailAddress = defaultValue={60 * this.props.default}
recipientEmailAddress; />
}, );
getRecipientEmail(){ }
return recipientEmailAddress; }
},
clearRecipientEmailAddress(){ The ’MinuteSlider’ component is configured with properties
this.recipientEmailAddress = null; to set the minimum, maximum, and default values for the
},
analysis time. The values are calculated in seconds, with the
The ’sendEmail’ function was implemented to send emails slider stepping in 60-second intervals to represent minutes.
to users containing the results of the malware analysis. If a The onInput event handler is used to capture the user’s input
recipient email address is provided, an email, complete with and update the state accordingly.
ENGR 489 (ENGINEERING PROJECT) 2023 10
<MinuteSlider They provide clear steps for users on how to submit a file
min={1} for analysis and what the different configuration options mean.
max={10}
default={10}
name=``analysisTime" D. Building drakcore Package and final deployment
onInput={this.handleInput}
/> After the web UI modifications were made the system
needed to be built and deployed again as the changes were on a
The ’handleInput’ function is implemented to manage the NodeJS development server and not on the actual DRAKVUF
state updates when the user adjusts the ’MinuteSlider’. The instance that was deployed. This process ensures that the
analysis time, represented in seconds, is stored in the compo- updated and enhanced web UI is packaged and ready for
nent’s state to be used during the sample upload process. deployment, allowing users to benefit from the new features
handleInput(event) { and improvements.
this.setState({ error: null }); During the build process, a significant challenge was en-
const field = event.target.name;
if (field === ``analysisTime") { countered. The SSL certificates of some external packages
this.setState({timeout: such as Minio and Python had expired, leading to issues
event.target.value}); in building the drakcore package. This obstacle required
}
} immediate attention to ensure the successful compilation and
packaging of the updated web UI.
The selected analysis time is displayed to the user in To address this issue, a meticulous review of the build
minutes, offering a clear and understandable representation of process was undertaken, and the problematic external package
the duration for which the analysis will run. links were identified. The expired SSL certificates necessitated
<small>{this.state.timeout/60} min</small> the modification of these links to ensure the seamless retrieval
of the required packages. The build process was guided by the
The implementation was rigorously tested to ensure that instructions provided in the official documentation for building
the slider responds accurately to user input and that the the drakcore package [12].
selected analysis time is correctly stored and utilised during the After the package was built we had to deploy the entire
sample analysis process. The feature was validated to ensure a DRAKVUF system again as per the steps outlined in Section
seamless user experience, accurately influencing the duration B of the implementation chapter.
of the malware analysis. Following this implementation and deployment, the
3) Virtual Machine Configuration: A modification made DRAKVUF system on the Dell Optiplex workstation was
to the web UI was the addition of the Virtual Machine transitioned to a more robust and accessible environment.
Configuration section. This is a static table that shows the The deployment was extended to the Engineering and Com-
available configurations of virtual machines. It pulls the details puter Science (ECS) school servers to enhance accessibility
from Xen and displays them as seen in Fig. 4 below. and performance. The web address for the deployment is
‘http://103.196.108.47:6300/’. This move was instrumental in
elevating the system’s availability, making it remotely acces-
sible to users. The integration into the ECS servers marked
a significant milestone, ensuring that the system’s capabilities
could be leveraged efficiently and securely.
Fig. 4: Available Configurations Screenshot The connection to the network switch was a pivotal aspect of
this deployment, ensuring network integration and data flow.
Since only one guest OS was implemented there is only one By being connected to the network switch, the DRAKVUF
configuration available. There are however multiple instances system was effectively positioned within the university’s DMZ.
available of that one guest OS. This strategic placement ensured that the system was not
4) Instructions: Instructions for the sample upload page only secure but also benefited from the enhanced network
were added. There was ambiguity around what was and was infrastructure, ensuring high-speed data processing and trans-
not needed for the analysis to run and what file types the mission.
system accepted. The instructions pose as a guide to help new
users how to navigate and use the upload sample page. VIII. E VALUATION
The instructions can be viewed in Fig. 5 below. The methods used to assess this project’s effectiveness are
presented in this section. This covers the survey’s methodology
and questions. Following the data gathering, this section ad-
dresses the project’s applicability and the solution’s measured
effectiveness.
A. Survey
A survey was undertaken to measure usability and user
Fig. 5: Submission Instructions Screenshot satisfaction when performing malware analysis through the
ENGR 489 (ENGINEERING PROJECT) 2023 11
DRAKVUF Sandbox. Eight individuals undertook the survey. comments and feedback regarding the proposed system, which
All individuals within the survey pool were associated with gives insight into possible future developments.
the Owhiti Cyber Security group, which included students and 2) Survey Results: Eight individuals completed the survey,
academic researchers. Participants were emailed the survey with the overall trend being that they were satisfied with the
specifications, which included the steps required to gain access tool’s usability and the information it provided.
to submission samples if they did not have any on their The subjects’ feedback on usability was generally positive.
computer alongside a link to the DRAKVUF Sandbox web A majority expressed that submitting a file for analysis on
UI. The subjects were then asked to access the tool and fill DRAKVUF was straightforward, attributing their positive ex-
out a web survey provided via a Microsoft Form. perience to the intuitive design and user-friendly interface.
1) Survey Questions: The survey consisted of nine ques- The ease of locating and selecting additional configuration
tions that asked users about their prior knowledge and about options was also highlighted, with most participants agreeing
the tool’s usability. It encouraged individuals to use the tool or strongly agreeing on its simplicity.
to perform analysis and extract information which enabled The email notification system received positive feedback.
them to answer specific questions. The questions asked and Five participants received the email notifications in their inbox,
the reasoning behind them are summarised below: three did not receive any. Upon further inspection however,
1. Have you ever used an online malware sandbox it revealed that the users that did not receive a notification
before? This question targeted an evaluation of the subject’s did not enter their email address into the input box, meaning
prior knowledge on sandboxes and if they were familiar with the system functioned appropriately. However, for any future
how to submit samples and the common configuration settings surveys, the point to provide an email address should be
on them. emphasised.
2. Do you know how a malware sandbox works? This Participants provided valuable insights on potential en-
question targeted an evaluation of the subject’s prior knowl- hancements. Suggestions included the incorporation of more
edge on sandboxes and if they were familiar with the inner- detailed explanations of features and a progress bar to inform
workings of them. users of the analysis status.
The following questions were choice questions, they The consistency in the ‘look and feel’ of the sandbox
spanned from Strongly Disagree to Strongly Agree and as- throughout the process was generally well-received, with most
sessed the usability, effectiveness and feel of the web UI. participants agreeing or strongly agreeing on its consistency.
3. It was easy and intuitive to submit a file for analysis on However, one participant noted that the graphics for the
DRAKVUF This question targeted an evaluation of the tool’s behavioural graph did not align with the overall UI design,
usability to gauge if it was too complicated and required more suggesting an area for aesthetic enhancement.
explanation or if users could understand the different features The feedback from the eight participants provides invaluable
and perform analysis. insights for the refinement of the DRAKVUF web UI. The
general satisfaction with the tool’s usability is encouraging,
4. It was easy to find and select additional configuration
while the identified areas for improvement offer clear path-
options (eg: Analysis Time, Modules enabled, Email input,
ways for enhancement.
etc.) This question tests a subject’s ability to find and select
Although the survey had a small pool of participants re-
additional configuration options on the web UI.
garding usability testing, eight participants allowed us to find
5. It was easy to find and view/download the logs and
almost as many usability problems as you’d find using many
reporting artefacts created by the sandbox This question
more test participants [13].
targets the evaluation of the tool’s reporting feature and if sub-
jects were able to easily find the artefacts and view/download
them. B. Functional Evaluation
6. The sandbox ‘look and feel’ was consistent throughout The DRAKVUF web UI, deployed on the ECS servers
the whole process This question targets the evaluation of the on Friday 22nd September 2023, has been tested to ensure
tool’s ‘look and feel’ and if the subjects found it consistent its functionality, reliability, and efficiency. The web UI has
throughout the process. proven to be a robust tool, consistently delivering accurate
7. Did you receive an email for the sample file submission and comprehensive results without any failures or significant
results in your inbox or junk mail? (After inputting your issues since deployment.
email in the ‘Customise’ options) This question aims to find There have been more than 30 sample submissions to the
out whether the email was received in the Inbox or Junk folder web UI since deployment, without failure. Compared to the
(or not received at all), to see if there are any bugs with this Cuckoo system that the Owhiti Cyber Security group currently
feature. uses this is a dramatic increase in reliability. My supervisors
8. Is there any new functionality you would like to have indicated that the Cuckoo system is notorious for crashing
be added to this system? We can use this information to even while it is not in use. DRAKVUF has been active for
provide further features based on the information provided by three weeks now on the ECS servers and has not failed nor
the subjects. crashed.
9. Is there any other feedback/suggestions you would A detailed examination of one of the sample submission
like to provide? This question provides space for additional instances provides a comprehensive insight into the system’s
ENGR 489 (ENGINEERING PROJECT) 2023 12
ACKNOWLEDGMENTS
I would like to thank my two supervisors Masood Man-
soori and Lisa Patterson for providing support throughout the
project. It would not be in the current state without their
help. I would also like to thank Muhammad Shabbir Abbasi
for his support with the multiple OS feature. Although the
implementation was not stable; it provided insights into the
workings of the system.
ENGR 489 (ENGINEERING PROJECT) 2023 14
A PPENDICES