Capability benchmarks ask what an agent can do. Safety red-teaming asks whether it says something harmful. I work on the question in between: whether a deployed agent complies with the policies it was given, across many turns, hidden tool calls and downstream state.
Policy violations are hard to see. They emerge after several benign turns, hide inside tool calls the user never sees, and surface only in the state of some downstream system. A static test suite covers a few of them. A human red-teamer finds more, slowly. Neither scales to the number of agents being deployed.
My PhD at MBZUAI, advised by Salem Lahlou and Nils Lukas, is about building auditors for that gap: learned agents that interact with a target system the way a real user would and search efficiently for the places where it breaks its rules. I care about coverage over raw hit counts, about auditors that stay plausible, and about verification that holds up when the target is far larger than the auditor.
Before this I spent four years on the physical side of robustness: adversarial patches on vision models, defenses that survive more than one patch, and a real-time testing framework for autonomous-vehicle perception that we validated outdoors. It is the same question at a different layer of the stack. How do you find the failure before deployment does?
PROBE (Policy Rule Observation via Behavioral Exploration) frames auditing as a partially observed Markov game between an auditor and a target agent with private tools and state. The auditor is trained with RL against severity-weighted verifiers, rewarded for distinct violations, progress into tool-relevant states, novelty and plausibility. A 4B auditor trained this way covers more policies and finds more violations than a zero-shot 30B adversary.
Treating a multi-agent system the way a fuzzer treats a program: generate inputs that push it into rarely visited states, watch for invariants that break, and learn where to look next. The interesting failures are the ones no single agent causes on its own: they appear when agents, tools and memories interact.
Finding one failure is easy. Finding all the different ones is the hard part. GFlowNets sample in proportion to reward, which makes them a natural fit for generating diverse test cases and auditor behaviours. Work here includes loss-guided auxiliary agents that counter mode collapse, GFlowNet-finetuned LLM agents for diverse symbolic music (Flowphony), and graph environments contributed to torchgfn.
AVATAR evaluates autonomous-vehicle perception against adversarial patches in real time, in CARLA and outdoors, with a KDE-based detector for adversarial vehicles. Earlier work covered multi-patch attacks, naturalistic support artifacts that raise model confidence, image resurfacing as a model-agnostic defense, a survey of 100+ patch attacks and defenses, and a membership-inference analysis of spiking neural networks.
Agentic AI auditing and safety; intelligent fuzz verification of multi-agent systems.
Privacy risks of spiking neural networks under membership inference; graph environments for torchgfn; GFlowNet-finetuned LLM agents for diverse melody generation.
Adversarial test features for AVL Scenius; a CARLA-based real-time test framework for object detectors; a KDE-based classifier for adversarial vehicles; outdoor validation experiments.
Surveyed 100+ physical adversarial threats; designed multi-patch attacks against state-of-the-art defenses; a model-agnostic, total-variation-based defense that works in a single scan.
Pricing surveys and statistical models for a US medical-device launch, in R, Python and Tableau.
Data-driven modelling and pitch/yaw control of a small wind turbine for maximum power point tracking.
Sun-sensor prototype and reaction-wheel control for a solar-sailing nanosat in low Earth orbit.
Research: agentic AI auditing and safety, intelligent fuzz verification of multi-agent systems.
Thesis: Towards Safeguarding Convolutional Neural Networks Against Physical Corruptions, with Apurva Narayan and John Braun.
Thesis: Hardware Implementation of Model Predictive Controller, with D. N. Sonawane.
Weekly lab sessions and tutorials on data analytics in R and Python.
Weekly labs, grading, and help designing lab tutorials.