Information that cannot be discussed during the event:

• Classified information
• Personally Identifiable Information (PII)
• Controlled Technical Information (CTI)
• Department of Defense (DoD) only information
• Government only information

Examples of information that can be discussed during the event:

• High–level conceptual information
• Controlled Unclassified Information that is not CTI, PII or for government only
• Public information from industry, academia, research institutions, etc.
• Information that has been approved for public release
• Otherwise uncontrolled and unclassified information

Key Outcomes from WG and Kick-Off Event:

Information Sharing

Community Awareness

Networking/Collaboration Opportunities

Initialize the Adversarial Artificial Intelligence Working Group

Other A2IWG information can be found in documents posted on this event website: https://a2iwg.ida.org/

Microsoft Teams and Phone Line information will be sent out prior to the event to those that have registered.

• Robustness Metrics/Certifications

• AI Trustworthiness

• Privacy and Confidentiality

• Test and Evaluation

• How should these results be certified?

• Is there a one–size–fits–all approach to metrics or will there be many approaches? If there are many approaches, how will the appropriate one be selected?

• Subgroup discovery — data mining technique — a finer grained analysis on models and their features — avoid using high level metrics such as accuracy

• What specific questions are test and evaluation teams going to need answered?

• How does this relate to explainable AI? Improved understanding of how features and labels map to each other will make it easier to predict what type of A2I/AML risks may be present.

• How does this relate to ethical AI? Possibility to leverage metrics

• Probability of attack success? (Pseudo–Non–Deterministic models vs Static–Deterministic Models)

• Results that guide stakeholders into making effective decisions. Do not want to steer them away from good products when the risk is acceptable.

• How should this be factored into the Authority to Operate (ATO) process? Should products need to be reassessed when this is added?

• Impact on decision support

• Assessing software supply chain — Software composition analysis (Github repos, Docker/Kubernetes containers) (Tampering with math/algorithms such as optimizers)

• Which A2I/AML methods should be used to assess a model? Should there be specific methods to assess the effectiveness of various defenses?

• Assessing data supply chain (Source of data, trustworthiness of source, data provenance) (Chain of custody throughout entire machine learning workflow/life–cycle)

• Defense through obscurity is not viable

• Algorithmic Robustness

• Adversarial Training

• Label Smoothing Defense

• Feature De–noising

• Ensemble Classifiers Using Different Features

• NIST 8269 — A Taxonomy and Terminology of AML
  https://csrc.nist.gov/publications/detail/nistir/8269/draft

• Secure System Engineering

• Monitor and Track Requests

• Monitor Streaming Data

• Obfuscate Predictions

• Knowledge framework for modeling threats

• Knowledge framework for modeling potential impact and risk to systems

• Assessing data supply chain (Source of data, trustworthiness of source, data provenance) (Chain of custody throughout entire machine learning workflow/life–cycle)

• An example from industry: Microsoft Blog — Threat Modeling AI/ML Systems and Dependencies
  https://docs.microsoft.com/en–us/security/engineering/threat–modeling–aiml

• AI model characteristics exploited by adversarial attacks

• Model differentiability

• Feature Selection

• Richness of model output

• Transfer learning

• Adversarial Robustness and explainable AI

• Adversarial examples are bugs or features?

• Gap between human and machine perception

• Adversarial AI for supervised approaches (Mullin)

• Feature Collision

• Convex Polytope

• Clean Label Backdoor

• Hidden Trigger Backdoor

• Adversarial AI for anomaly detection approaches

• Self–supervised learning (e.g., NLP)

• Unsupervised learning (e.g., Clustering)

• Evasion

• Data poisoning

• Adversarial AI for Reinforcement Learning

• Dependent on simulated environments

• Influence action path of RL system

• Adversarial policies

• Model Decay vs. Model Attack

• Online Learning

• A/B Testing

• Real world attacks — model or hardware

• Electromagnetic

• Visual spoofing

• Anomaly normalization

• Risk Evaluation & Mitigation of model failure

• Ensemble

• Hidden models

• Orthogonal mode pairing