Experimental Workflow for AVATAR

1University of Waterloo, 2Western University, 3AVL, Graz Austria
*Indicates Main Author, Indicates Equal Advising
Inference Overview

AVATAR is designed to standardize the testing process for adversarial patches, a significant concern in autonomous driving scenarios. The figure illustrates the overall flow diagram of the AVATAR Framework

🎯 High-Level Overview of Workflow

🚗 Installing CARLA from Source 🛠

Installing CARLA from source allows full access to its underlying code and assets, enabling customization of maps, vehicle models, and other resources. It provides the flexibility to modify and extend the simulator's functionality, such as adding new features or adjusting existing ones, which cannot be done with pre-built versions.

  • ➤ As we need to integrate adversarial patch in the CARLA map, we need to install CARLA from source.
  • ➤ This process is slightly more complex as we also need to install Unreal Engine (UE), as compared to using pre-built CARLA.
  • ➤ Ensure that the GPU drivers and CUDA dependencies are properly configured, ensuring compatibility with PyTorch version.
  • ➤ Follow step-by-step instruction from this link CARLA Linux Build to install Unreal Engine and CARLA.
  • ➤ Eventhough we have newer versions of CARLA as well as UE, we work with stable versions: CARLA 0.9.12 and UE 4.26.
  • ➤ It is essential to install CARLA's fork of UE 4.26, as it includes specific patches required for CARLA. Avoid using a random version to ensure compatibility.

🤖 Training Object Detection Model

We utilize YOLO for our experiments due to its superior performance compared to R-CNNs. YOLO (You Only Look Once) offers faster inference and efficiency by processing the entire image in a single pass, making it ideal for real-time applications

  • ➤ AVATAR is object detection model-agnostic but requires the model's weights and architecture for its use with CARLA.
  • ➤ The model's architecture is crucial for model loading for object detection using CARLA's Python API Client (see later).
  • ➤ The model's weights can be imported during experiment initialization using AVATAR's GUI.
  • ➤ Ensure the same technique (code) used to train or load the model is followed when integrating it into CARLA.
  • ➤ YOLOv5 was used in our experiments; however, newer architectures (e.g., YOLOv11) can also be utilized.
  • ➤ Refer to Ultralytics GitHub to load and train an object detection model of your choice.
  • ➤ The dataset for model training must be photorealistic like CART for its effectiveness in CARLA or in the real world

🕷️ Training Adversarial Patch

Download a pre-trained adversarial patch, crafted using the CART dataset, for attacking YOLOv5's detection of a car in the CARLA environment. We use this patch in all our experiments.

adv patch
Download Image File
  • ➤ Adversarial patches can be designed as targeted (more challenging) or untargeted (simpler).
  • ➤ In our experiments, the patch misled YOLOv5 to ignore cars with the patch applied (untargeted approach).
  • ➤ Refer to this GitHub repository for training adversarial patches on YOLOv5.
  • ➤ The YOLOv5 model in the repository is based on Ultralytics, making it compatible with our trained models.
  • ➤ Patch training hyperparameters require trial & error. (Our case: 200×200 patch, 100 epochs, learning rate 0.03).
  • ➤ Use a photorealistic dataset like CART for effective results in CARLA or real-world scenarios.
  • ➤ Traning combines classification loss, total variation, and non-printability score, applying patch randomly over each image.

📸 Object Detection in CARLA

CARLA's Python API Client is an interface that enables users to interact with simulator programmatically using Python. It provides a high-level API to control various aspects of simulation, like spawning, managing or controlling actors and accessing their sensor data. This client facilitates seamless integration of custom code. Hence, the object detection must be integrated into the Python API client. This allows the vehicle to autonomously navigate the map while detecting objects in the scene.

adv patch

Source: CARLA Python API Docs

  • ➤ CARLA, built from source, requires substantial GPU resources due to its dependency on Unreal Engine (GPU-intensive).
  • ➤ In our experiments, the simulation runs in autopilot mode, enabling the vehicle to navigate autonomously within the map.
  • ➤ Combining object detection with Python API's autopilot functionality demands a high-performance GPU (min. 16GB).
  • ➤ Access the object detection integration code for Python API's autopilot mode here: GitHub Link.

⚠️🚗 Adversarial Patch in CARLA

Allows evaluation of the adversarial patch's impact on object detection, while blending the patch visually with the scene for realism. It enables testing autonomous driving systems in near-real-world conditions.

material blp
vehicle blp

Material Generation and Modified Adversarial Vehicle Blueprint

  • Adversarial Patch Import into CARLA:
    • Import the adversarial patch into Unreal Engine. Convert the trained patch image (.png/.jpeg) into a CARLA asset (.uasset). The patch can be attached to a moving actor (vehicle) or placed at a static location in the map.
  • Material Instance Creation in Unreal Engine:
    • Create a material instance in UE. Configure properties such as opacity: controls transparency and roughness: adjusts surface texture. Use metallic material with high roughness and opacity for better visibility. Assign the trained patch's file path in UE’s blueprint library. UE reads the image and converts it into a material instance.
  • Applying Adversarial Patch as a CARLA Asset:
    • ▶ Adversarial material instance created in UE can be applied to any CARLA asset, such as a vehicle or a static object.
    • ▶ This integration ensures the patch appears as a natural and realistic part of the environment.
    • ▶ Example setup: Attach the patch to a board mounted on a vehicle that moves randomly across the CARLA map.
    • ▶ This dynamic setup allows for evaluating the impact of the patch on object detection during autonomous driving scenarios.
    • ▶ A custom blueprint for the adversarial patch board is created in UE to enhance flexibility.
    • ▶ The blueprint enables the selection of different patches dynamically, allowing experimentation with various designs.

🎛️ AVATAR's GUI

The GUI serves as the primary entry point for interacting with the AVATAR framework. It seamlessly integrates with the underlying PythonAPI, automatically invoking the components with relevant parameters to start a PyGame window simulation displaying the scene in CARLA.

  • ➤ The features of GUI is explained in detail in the main paper.
  • ➤ Access the Python (Tkinter based) code for the AVATAR's GUI here: GitHub Link.

Typically, the PythonAPI is sufficient to interact with CARLA, eliminating the need to directly work with Unreal Engine. In our case, we modify CARLA assets (e.g., for adversarial patch attacks). This requires access to Unreal Engine asset files to specify whether an adversarial patch attack should be conducted and to select the patch image to use.


Within the AVATAR GitHub repository, there is a patches folder. Inside, there is a flag file used to signal Unreal Engine whether an attack should be performed, and a patch image file is transferred to Unreal Engine, converted into an material asset, and applied to a vehicle asset.


NOTE: Do not delete the flag file, as it is critical for attack execution. one can replace the patch image with a custom image to be used as adversarial patch in simulations.

⚙️ Post-Simulation Adversarial Trace Classifier

The Adversarial Trace Classifier (ATC) is developed for post simulation analysis to robustly differentiate between original vehicle behavior and the behavior affected by adversarial patches.

    • ➤ The Adversarial Trace Classifier (ATC) algorithm is thoroughly detailed in the main paper.
    • ➤ ATC is designed to minimize human bias in adversarial analysis and automatically flag adverarial presence during simulations
    • ➤ During simulations, the target vehicle's detection confidence score are recorded and stored in results folder of the repository.
    • ➤ The ATC analyzes these confidence score traces to identify adversarial activity during simulations.
    • ➤ Access the Jupyter Notebook code for implementing ATC here: GitHub Link.

BibTeX

If you find our contribution useful, please consider citing our work.

@article{sharma2024avatar,
      title={AVATAR: Autonomous Vehicle Assessment through Testing of Adversarial Patches in Real-time},
      author={Sharma, Abhijith and Narayan, Apurva and Azad, Nasser Lashgarian and Fischmeister, Sebastian and Marksteiner, Stefan},
      journal={IEEE Transactions on Intelligent Vehicles},
      year={2024},
      publisher={IEEE}
    }

Acknowledgements

We extend our gratitude to the members of the Real-time Embedded Software Group @UWaterloo for their invaluable insights, critical brainstorming sessions, and innovative ideas that greatly contributed to this work.