DomainForensics: Exposing Face Forgery across Domains via Bi-directional Adaptation
近年来，DeepFake检测方法在公共数据集上表现出优异的性能，但在新伪造物上明显退化。解决这个问题非常重要，因为随着不断发展的生成技术，每天都会涌现出新的伪造物。为了解决这个问题，我们在无监督域迁移角度上重新思考，并提出了一种新的解决方案。我们的解决方案名为DomainForensics，旨在将已知的伪造知识传递给新的伪造物。与最近的努力相比，我们的解决方案不关注数据视图，而是关注DeepFake检测器的学习策略，通过跨域差异的alignment传递新伪造物的知识。特别是，与考虑语义类分类的一般域迁移方法相比，我们的方法捕捉到细微的伪造痕迹。我们描述了一种新的双向域迁移策略，专门用于捕捉跨域的伪造知识。具体来说，我们的策略考虑了正向和反向迁移，在正向迁移中，我们在源域对DeepFake检测器进行监督训练，并共同应用对抗特征迁移，将已知伪造物中提取出的检测能力传递给新的伪造物。在反向迁移中，我们通过将对抗迁移与自监督学习相结合，进一步提高了知识传递。这使得检测器能够从未标记数据中揭示新的伪造特征，并避免忘记已知知识。
Recent DeepFake detection methods have shown excellent performance on public datasets but are significantly degraded on new forgeries. Solving this problem is important, as new forgeries emerge daily with the continuously evolving generative techniques. Many efforts have been made for this issue by seeking the commonly existing traces empirically on data level. In this paper, we rethink this problem and propose a new solution from the unsupervised domain adaptation perspective. Our solution, called DomainForensics, aims to transfer the forgery knowledge from known forgeries to new forgeries. Unlike recent efforts, our solution does not focus on data view but on learning strategies of DeepFake detectors to capture the knowledge of new forgeries through the alignment of domain discrepancies. In particular, unlike the general domain adaptation methods which consider the knowledge transfer in the semantic class category, thus having limited application, our approach captures the subtle forgery traces. We describe a new bi-directional adaptation strategy dedicated to capturing the forgery knowledge across domains. Specifically, our strategy considers both forward and backward adaptation, to transfer the forgery knowledge from the source domain to the target domain in forward adaptation and then reverse the adaptation from the target domain to the source domain in backward adaptation. In forward adaptation, we perform supervised training for the DeepFake detector in the source domain and jointly employ adversarial feature adaptation to transfer the ability to detect manipulated faces from known forgeries to new forgeries. In backward adaptation, we further improve the knowledge transfer by coupling adversarial adaptation with self-distillation on new forgeries. This enables the detector to expose new forgery features from unlabeled data and avoid forgetting the known knowledge of known…
https://arxiv.org/abs/2312.10680
https://arxiv.org/pdf/2312.10680.pdf