Tech Reports

Cybersecurity incident response is an increasingly complex task that requires humans to quickly analyze large amounts of data and make optimal decisions. One solution is to train autonomous cyber defense agents using reinforcement learning techniques to provide faster and more accurate responses. In this thesis, I investigate the effects of reward shaping and the use of human input during training. Specifically, I present various modified reward functions and their effects on the agent's behavior. I also explore kickstarting the agent's training with human input and demonstrate how this can influence the agent's performance. This thesis provides preliminary work in these directions.

Brain decoding is the process of reconstructing stimuli by interpreting brain activity. In particular, given brain signals recorded using functional magnetic resonance imaging (fMRI), we want to determine the language that incited these brain responses. Previous research has successfully demonstrated reconstruction of word sequences from these fMRI recordings, but this does not capture the more refined aspects of speech, such as intonation, emotion, and intent. In this project, our aim is to improve the performance of brain decoding by reconstructing audio segments that recover the acoustic properties of perceived speech.

Recent advances in hardware acceleration and sampling techniques have made implementing photorealistic techniques like soft shadows and realistic reflections/re- fractions feasible for real-time applications like modern video games, which have hard performance requirements (usually in the tens of milliseconds). Previously, these applications used ad-hoc techniques like rasterization, screen-space methods, and baked lighting to accomplish photorealistic imagery with major visual compro- mises. In some contexts, a user’s non spatially uniform perception of detail means that photorealistic results aren’t required in every detail of the frame (such as at- tention on the car vs the overall environment in a modern racing game). Using this insight, we varied which models went through which pipeline (rasterization/raster or hardware accelerated ray tracing/RT) based on perceived visual importance to increase performance. This resulted in overall speedups of up to 4-10ms per frame.

Linux uses a load balancer to distribute running tasks across multiple cores in a system. Ideally, the load balancer is work-conserving, which means that it does not leave any core idle. Additionally, the load balancer should evenly distribute work between the cores to maximize the progress made on each task. Currently, the best approach to testing if a particular implementation satisfies performance requirements is using fuzzing, which can miss certain worst-cases. On the other hand, formal verification of the scheduler is infeasible due to the large size of the code base, and the rapid rate of development would require constant updates to the model. This work aims to reach a middle ground that allows us to generate cases where the load balancer makes an incorrect decision without modeling the entire scheduler. We implement a tracing module in the kernel that collects the visible state, i.e. the minimum set of information the load balancer uses to make its decision. There also exists hidden state that the heuristic does not directly measure, which may affect the performance of the system but does not affect the load balancer's decision. This allows us to manipulate the hidden state to "hallucinate" examples where performance suffers even if such a case is not observed. To do this, we encode consistency checks that ensure the generated hidden state agrees with the visible state and invariants that formally encode desired performance guarantees. Using SMT solvers, we can synthesize hidden states that satisfy the consistency checks but violate an invariant.

Aging and physical activity are associated with structural and functional changes in the brain and are prominent risk factors for predicting the onset of neurodegenerative diseases. Previous literature shows that the brain undergoes physical changes in response to aging, such as reductions in brain volume, decline in white matter integrity, and abnormalities in brain activity. Engaging in physical activity is known to increase neurogenesis and promote synaptic plasticity, slowing down neuronal damage and degeneration leading to increases in perfusion and increases in gray and white matter volumes. Traditional approaches to study brain aging rely on structural measures with limited investigation of brains hemodynamics, which may diverge from normal aging prior to structural changes. We hypothesize that measures of brain function and hemodynamics can be used to calculate a cerebrovascular brain age and the relation between cerebrovascular brain age and chronological age will be affected by physical activity and exercise. The presented work characterizes the effects of physical activity and exercise on brain function and cerebrovascular resilience using a 3D convolutional neural network based on MRI measures of cerebrovascular reactivity and amplitude low level fluctuations to estimate brain age. Two models will be developed to determine how physical fitness metrics affect the model’s performance.

Probabilistic programs have a variety of applications in randomized algorithms, cryptography, and distributed systems. However, reasoning about and verifying such programs, especially in a modular way, is difficult. Recent work has used probabilistic independence as the basis for a new model of separation that allows separation-logic-like reasoning for probabilistic programs. However, as these probabilistic program logics develop, so has the complexity of their underlying measure theoretic models. This paper uses the Rocq proof assistant to formalize the definitions and properties of probability spaces as resources, with the independent product operation as composition. We mechanically prove that probability spaces satisfy the requirements of a Kripke resource monoid, and in doing so take the first steps towards mechanizing a flexible, trusted program logic for probabilistic programs.

Dynamic environments present significant challenges for instruction-following navigation algorithms, particularly when replanning is required to adapt to the changing environment. Existing solutions often focus only on one aspect of the problem, either addressing instruction following in static settings or using social navigation in dynamic environments without considering explicitly given instructions. Current instruction-following solutions often rely on vision-language models (VLMs) to generate cost maps as direct inputs for conventional optimization-based planning methods. While these methods work well when planning time is not critical, the computational requirements of current VLMs often make them impractical for more variable dynamic environments. This work is produced as part of a larger project that proposes a two-step approach to this problem. First, train a potential-based diffusion model to generate collision-free paths to a specified goal. Then, fine-tune separate versions of this baseline model, each tailored to generate paths aligned with a corresponding atomic instruction. In this work specifically, we present three main contributions to this project. First, we outline a method to generate random viable paths as effective training data for the diffusion model. We also create simple functions that evaluate task completion and show that fine-tuning using these functions can modify the same initial model to adhere to different instructions. Finally, we show that composing multiple fine-tuned models can produce paths that follow more complex instructions unseen at training time. Some background about the initial diffusion model and fine-tuning approach is provided for context, but it was designed and implemented in the larger project, not this work. Our approach as a whole enables robots to follow instruction-grounded plans in dynamic scenarios without relying on VLM inputs. Furthermore, we demonstrate that fine-tuned models for individual instructions can be composed to produce plans that replicate more complex behaviors.

Softmax attention incurs quadratic time complexity in sequence length, making it the primary bottleneck in transformer training and inference for long sequences. Structured Masked Attention (SMA) offers a sub-quadratic alternative by removing softmax normalization and applying a structured matrix as the attention mask. Dropping softmax enables reformulating elementwise masking as a matrix multiplication, allowing the use of fast transform algorithms to compute the operation in sub-quadratic time when the mask is structured. Implementing SMA for a given structured mask is straightforward if a standalone fast transform implementation is available. However, standard implementations of these transforms often achieve low hardware utilization on GPUs because they do not leverage high-throughput matrix-multiply units like Tensor Cores. Furthermore, using a separate kernel to perform the transform prevents operator fusion, missing critical opportunities to reuse data in on-chip SRAM. In this work, we address these challenges for SMA when using Walsh-Hadamard and discrete Fourier transform matrices, both of which admit O(n log n) algorithms for computing matrix-vector products. Specifically, we design fused, Tensor Core accelerated kernels to efficiently perform inference and backpropagation for Hadamard and Fourier masked attention while retaining sub-quadratic time complexity.

Learning to automate the geometric reassembly of broken 3D objects is a complex task recently tackled by the scientific community with various practical applications, such as bone and artifact restoration. Recent works have shown that learning-based methods can reassemble fractured objects using local geometric information without requiring knowledge of the original global shape prior. These approaches struggle with objects involving complex fractures involving many or tiny fragments that lack sufficiently descriptive local geometric information, and also often assume that all fragments are present, which does not reflect real world scenarios where fragments may be missing. This paper introduces SHARD, a template-based reassembly framework that aims to reconstruct broken objects by registering fragments to an inferred global shape prior. Our local-global alignment framework mainly consists of four components: 1) a shared global-fragment geometric point cloud superpoint and feature extractor, 2) an efficient high-order convolution layer to refine and compare features of pairs to be matched, 3) superpoint matching to identify reliable coarse-level correspondences between refined global and fragment superpoint features, and 4) a fine-level registration step for improved fragment registration alignment. We evaluate SHARD on the Breaking Bad dataset and achieve promising reconstruction results through this template-guided framework, and explore potential improvements and tradeoffs in reassembly accuracy and generalizability.

Modern constraint solvers such as Z3, Gurobi, and CP-SAT have been effective in solving many different classes of problems, including program verification, resource scheduling, and pathfinding. Given their flexibility and robustness, we seek to utilize constraint solvers for encoding and generating meshes. To our knowledge, previous research in this area typically focuses on using linear and quadratic solvers for optimizing local geometric criteria or parameterizations. Our work deviates from the established literature by exploring the possibility of generating entire meshes defined solely by constraints. In order to accomplish this, we characterize several different techniques for defining global mesh topologies and geometries via combinations of logical, linear, and nonlinear constraints. Using satisfiability modulo theory (SMT) and boolean satisfiability (SAT) solvers on these constraint systems, we generate meshes that comply with given properties, such as being manifold and non-self-intersecting. Finally, we contrast the empirical performance of different implementations and explore their feasibility. In doing so, we identify why certain mesh constraint formulations can significantly impact solver performance and complexity. These results could provide an avenue to further explore new methods for mesh generation and simplification.

Machine learning-based generative protein models offer a way to design novel proteins with useful properties, with applications in medicine, plastic waste decomposition, and material design. In just the last few years, diffusion models which generate protein backbones have shown immense promise. Model development relies on metrics to evaluate architectures and hyperparameter choices, but current in silico metrics such as designability are both inconsistently implemented and fundamentally lacking. I propose a new set of metrics based on analyzing protein substructures known as domains, and I evaluate Genie2, Proteus, RFDiffusion, and Chroma with them. My metrics show that modern generative models do not produce realistic domains, and as a result offer new tools to guide model development.

Computational methods provide a powerful framework for understanding complex and high-dimensional feature spaces, such as the human brain. Prior studies have shown that neural networks, specifically self supervised learning (SSL) models such as WavLM-large, can be reliably used to predict cortical responses and selectivity patterns in well characterized regions involved in the speech processing hierarchy. These models provide a framework for understanding the relationship between language and neural responses, enabling further insights into the roles of different brain regions in the speech processing hierarchy. Building upon this success, we apply these methods to investigate a less understood cortical area: the superior part of the ventral premotor cortex (sPMv). Located anterior to the central sulcus and superior to the primary mouth motor region (M1M), sPMv has been implicated in all levels of the speech processing hierarchy and in both speech production and perception. However, its precise role and structural organization remain unclear. In this work, we model sPMv’s responses to natural speech and, using methods such as principal component analysis (PCA), K-means clustering, and probing, analyze feature spaces associated with different levels of the speech processing hierarchy. We identify an anterior-posterior gradient of speech feature selectivity extending from sPMv to the superior part of M1M. This gradient mirrors patterns observed in well understood speech processing areas such as the auditory cortex and reveals distinct subregions within sPMv specialized for different levels of speech processing. These findings demonstrate how computational tools such as SSLs can be used to gain insight into less understood regions of the human cortex and help explain why sPMv is involved in all levels of the speech processing hierarchy.

We present a computational model for simulating rigid rods under relativistic physics. Although rigid bodies are often seen as incompatible with Special Relativity, we develop a formalism that enables a physically grounded simulation. First, we propose a continuous-time Lorentz covariant formulation of Hamilton's Principle in a lab frame and produce a relativistic Lagrangian. The Lagrangian contains spring potentials precisely defined in a COM frame, which is the special instantaneous inertial frame in which the spring endpoints are mass-weighted equidistant. In order to both keep track of this COM frame and determine the lab time events corresponding with this frame, we introduce additional constraint terms to our Lagrangian. Particularly, due to the relativity of simultaneity, endpoint events at a given snapshot of lab time are not usually simultaneous within the COM frame. To address this mismatch, we introduce time shift variables and a corresponding constraint. From there, we discretize the action, yielding a spring-particle system governed by Euler–Lagrange equations at snapshots of lab time, using Lagrange multipliers within a constrained Hamilton's Principle. Finally, we take this system to the infinite stiffness limit to model rigidity, and examine the results.

The use of randomness in quantum circuits is an intrinsically interesting property due to the model's convergence to the hard-to-simulate Haar measure. While random circuit sampling is both promising for quantum advantage and realistically implementable on a NISQ device, it is not yet efficiently verifiable. Previous work in studying peaked random circuit sampling models has shown optimism for a potentially viable model. In this paper, we extend those observations by studying a simpler, alternative model to quantum computation involving beamsplitter networks. We present numerical and theoretical findings on the structure of peaked beamsplitter networks and evaluate their potential as a candidate for quantum advantage experiments.

The widespread success of artificial intelligence in fields like natural language processing and computer vision has not yet fully transferred to robotics, where progress is hindered by the lack of large-scale training data and the complexity of real-world tasks. To address this, many robot learning researchers are pushing to get robots deployed at scale in everyday unstructured environments like our homes to initiate a data flywheel. While current robot learning systems are effective for certain short-horizon tasks, they are not designed to autonomously operate over long time horizons in unstructured environments. This thesis focuses on addressing two key challenges for robots operating over long time horizons: memory and lifelong learning. We propose two novel methods to advance these capabilities. First, we introduce t-DGR, a trajectory-based deep generative replay method that achieves state-of-the-art performance on Continual World benchmarks, advancing lifelong learning. Second, we develop a framework that leverages human demonstrations to teach agents effective memory utilization, improving learning efficiency and success rates on Memory Gym tasks. Finally, we discuss future directions for achieving the lifelong learning and memory capabilities necessary for robots to function at scale in real-world settings.

Machine learning at the OS layer offers benefits such as improved decision-making in kernel subsystems, adaptability to workloads and hardware, and enhanced performance through efficient resource management. However, the integration of ML models can introduce increased complexity and additional performance overhead, potentially becoming a bottleneck. One approach to address this challenge is LAKE, a system that utilizes ML and specialized hardware like GPUs to handle the computational load associated with ML decisionmaking within the kernel space. LAKE builds upon LinnOS, an OS that employs a lightweight neural network for inferring SSD performance at a fine, per-I/O granularity. In this paper, we consider additional random I/O device redirection policies that extend both the baseline Linux I/O scheduler and the LAKE latency predictor. We use these random redirection policies to investigate the effectiveness of the Linux and LAKE I/O schedulers, evaluate the performance of all these policies on traces, and describe the benefits of random redirection algorithms.

In reinforcement learning (RL), an agent attempts to learn an optimal policy through repeated interactions with the environment. Unlike traditional reinforce- ment learning algorithms, which learn state-value estimates to arrive at an optimal policy, distributional RL (distRL) aims to model the entire state-value distribution. The primary advantage of distributional methods stems from the regularizing ef- fect of learning a distribution. Recent distRL models attempt to take advantage of the secondary benefits of modeling a distribution, typically by using properties of the distribution to guide exploration or by using distorted expectations over the distribution to create risk-averse or risk-seeking policies. Analysis of prior work suggests that exploration-based methods achieve impressive performance but often suffer from biased data collection, unstable convergence, and overestimation. On the other hand, methods that distort the return distribution to train risk-averse policies have better convergence properties but achieve sub-optimal returns on long-horizon problems due to insufficient exploration. Building off recent advances in exploration techniques within expected and distributional RL, we develop three methods that utilize the state-value distribution to guide exploration. Our work demonstrates how diverse concepts from traditional exploration techniques and risk-aware RL can be combined with knowledge of the value distribution to develop novel algorithms that fully leverage the capabilities of distRL. We evaluate our three proposed methods on a set of of MDP and sparse classic control tasks. Based on benchmark performance, we highlight the promising potential of the Separated QR-DQN (SQR) method and suggest ways it can be adapted to solve complex sparse reward tasks in future work.

As machines go from single-core to multicore, an important aspect of the kernel is load balancing. Load balancing is when the Linux scheduler tries to schedule work across multiple cores fairly, ensuring that no core has too little work or has too much work. Ensuring effective load-balancing logic is critical to taking advantage of the many cores a processor might have. Ideally, when there are more tasks than cores, no core should be idle, each core having work to do. However, that is sometimes not the case, and one CPU needs to “steal” work from another CPU, but for various reasons does not go down that route. It is important to verify that the load-balancing mechanism is working properly. In this paper, we explore the load balancing algorithm in the Linux kernel. We look at the logic that goes into load balancing, what metrics it looks at to determine what core to steal work from, and a possible method to determine whether or not there is a bug in the load balancer. It turns out that while there are many formal methods to discover a bug in the Linux kernel, there are other less formal ways to get a sense of whether or not there is an efficiency issue in the scheduler. The load balancer for the Linux kernel v6.8-rc1 seems to be working properly for certain configurations of a complex workload like the kernel compilation process.

Recent cache replacement policies take one of two approaches to assigning eviction priorities for cache lines. The first is a fine-grained, learning-based approach, exemplified by policies such as Hawkeye and Mockingjay, that inserts cache lines with a priority based on their past behavior. This learning associates behaviors with Program Counters (PCs), that is, the address of the load instruction. Unfortunately, propagating the PC through the memory hierarchy can be a prohibitive cost for chip designers in terms of both physical hardware overhead and the extensive time required to verify and validate these changes to the memory system. The second is a coarse-grained approach, exemplified by LRU, DRRIP [11], KPC (Kill the PC) [14] and RLR (Reinforcement Learning Replacement) [18], where lines are inserted either all with the same priority or one of several static priority values. While these approaches do not use the PC, it is much less accurate than learning-based approaches because it does not actually learn the reuse patterns of different lines. The reason that the program counter is so useful is because it has high separability, meaning that it is exceptional at grouping similar cache lines together, which allows for effective prediction of future behavior. This paper introduces a new cache replacement policy called RPC (Replace the PC) as the first fine-grained, learning-based replacement policy that does not use the program counter. To create RPC, we modify the Mockingjay policy to use a new signature that achieves similar line separability as the PC. RPC outperforms prior PC-free policies: For a set of SPEC 2006 [7] and GAP [3] benchmarks using the IPCP prefetcher [16], it achieves a speedup over LRU of 10.18%, as opposed to KPC which achieves 8.80% and RLR which gets just 1.66%. We also propose a modification to learning-based replacement policies, WARP (Writeback Access Reuse Prediction) that takes writeback accesses into consideration when learning. WARP can be applied to RPC to create WARP-RPC which achieves a speedup of 10.48% over LRU. Lastly we will introduce, WALK-MIN (Writeback Access LooKahead MIN), which builds on Bélády's MIN [5] and Demand-MIN [9] to modify the MIN policy to be write-back aware in a multilevel writeback cache hierarchy.

V-bed knitting machines are machines which can produce a variety of soft objects by knitting and maneuvering loops of yarn using hundreds of needles arranged in two beds, or rows. These machines are programmed using a set of low-level instructions, and many programs can have the same overall result while taking vastly different amounts of time to execute. Thus, it is natural to attempt to optimize knitting machine programs. One part of knitting a complex object is transfer planning. Transfer plans are sequences of instructions which move loops between the needles, arranging them for subsequent operations. Prior work on transfer planning includes polynomial time algorithms which solve special cases and a general A*-based search algorithm. We will improve this A* approach with search tree pruning based on observations about stacking loops on a single needle, mainly that loops on the same needle move in tandem and can't be separated. This pruned search space will also allow for a slightly simpler state representation.

With the exponential growth in unstructured data over the past few years, Graph Pattern Mining (GPM) serves as an important tool to yield new insights in a variety of domains, from biochemical compounds to social networks. GPM workloads generate massive amounts of intermediate state and require irregular data access patterns that are challenging to scale on typical architectures. PANDOHammer, an extension of the distributed, manycore Hammerblade architecture, introduces a new supercomputing system that can help serve large-scale graph processing workloads. With 1000s of cores that provide high throughput and latency tolerance, PANDOHammer exposes a rich, unexplored design space for graph processing workloads. We explore Triangle Counting, a simple, yet fundamental, GPM workload on the PANDOHammer system. We (1) create and compare new in-memory distributed representations of large graphs, (2) quantify the performance benefit of moving compute-to-data rather than moving-data-to-compute on PANDOHammer, and (3) investigate the optimal task decomposition for TC on PANDOHammer. Ultimately, we contribute new insights about extracting performance from the PANDOHammer supercomputing system. We find that running Triangle Counting by moving-compute-to-data with extremely fine-grained task decomposition on load-balanced distributed in-memory graphs outperforms Triangle Counting by chunking tasks, using complete vertex caching, and moving-data-to-compute. Not only do we contribute the first end-to-end graph pattern mining application on PANDOHammer, but we also provide a richer understanding of PANDOHammer that empowers us to develop more complex applications for this new supercomputing system.

Recent approaches assessing the capabilities of Large Language Models (LLMs) to generate support responses for negative emotions often focus on broad classifications of negative linguistic tone or style, rather than addressing the specific causal factors underlying these emotions. While the former strategy may provide temporary relief in the short-term, only the latter addresses the root causes and fosters long-term solutions. Psychology literature suggests that a precursor to emotion is the way a person appraises the situation surrounding it, signaling new dimensions that emotional support systems can reappraise toward sustainable, healthy emotions. To work towards this end, we build RESORT, a framework that can automatically elicit these cognitive reappraisals, leveraging language to instill reappraisal guidelines across multiple dimensions in LLM instructions. This thesis focuses on one of the strategies within the framework, RESORT-ITER, which iteratively incorporates reappraisal strategies in multiple relevant dimensions to build a cumulative emotional support response. We evaluate reappraisal ability of RESORT-ITER with expert psychologists, which demonstrates that LLMs even at the 7B scale are capable of effective long-term reappraisals. Our evaluation of the cumulative message shows that this proficiency does not entail significant reduction in perceived short-term emotional support by non-specialists.

Sparse graphs are prevalent in many scientific and real-world applications. Compressed sparse row (CSR), a sparse matrix representation, is commonly used to represent sparse graphs in memory for analytics workflows because it increases data locality and is more amenable to pre-fetching than other representations. However, a major limitation of CSR is its lack of support for efficient, in-place updates. In particular, due to its strict compactness, a CSR graph must be rebuilt entirely to add a new edge. Immutability is a critical limitation for graph analytics pipelines seeking real-time insights about evolving graphs. This paper proposes a novel data structure, the log-structured CSR (LSCSR), which maintains the performance benefits of CSR while enabling batched updates to the graph. We evaluate LSCSR on synthetic workloads and explore the trade-offs it makes between compactness, locality, and mutability.

Online transaction processing (OLTP) databases often suffer from reduced throughput, due to high contention on a small set of frequently accessed (”hot”) records. Recent in-network acceleration approaches propose housing hot records in the top-of-rack P4-programmable switch’s memory, and expressing transaction logic in lock-free, line-rate P4 code- thereby avoiding contention. Since switch memory is scarce, we want to reduce the database’s switch memory footprint at the smallest cost to throughput possible. Our solution is MINT, which consolidates the database’s interactions with records on-switch into a fraction of overall execution time (the hot period), saving switch memory during the rest of execution (cold period). This reduces total switch memory footprint by up to 10.7× on our YCSB workload, with only a modest 16% throughput drop.