Competing for pixels: a self-play algorithm for weakly-supervised segmentation
弱监督分割（WSS）方法，依赖于图像级标签表示物体存在，缺乏标签与感兴趣区域（ROIs）之间的明确对应关系，提出了一个显著的挑战。尽管如此，由于与完全监督分割相比，WSS方法的标注成本较低，WSS方法引起了人们的关注。利用强化学习（RL）自回归，我们提出了一种新型的WSS方法，将图像分割成两个代理之间的竞争。我们将分割定义为两个代理争夺选择包含ROI的补丁，直到所有此类补丁都被选择为止。每个时间步的得分，用于计算代理训练的奖励，代表物体在选择中的概率，由仅使用物体存在级二分类标签预训练的对象检测器确定。此外，我们提出了一种游戏终止条件，可以在所有ROI-包含补丁用尽时由任何一方调用，然后从每个补丁中选择最终的补丁。在终止时，如果ROI-包含补丁用尽或者由对手发现了一个ROI-包含补丁，则代理会被激励。这种竞争设置确保了WSS方法中的过度或不足分割问题的最小化。在四个数据集上的广泛实验表明，与最近的方法相比，WSS方法取得了显著的性能提升。代码：https:// this URL
Weakly-supervised segmentation (WSS) methods, reliant on image-level labels indicating object presence, lack explicit correspondence between labels and regions of interest (ROIs), posing a significant challenge. Despite this, WSS methods have attracted attention due to their much lower annotation costs compared to fully-supervised segmentation. Leveraging reinforcement learning (RL) self-play, we propose a novel WSS method that gamifies image segmentation of a ROI. We formulate segmentation as a competition between two agents that compete to select ROI-containing patches until exhaustion of all such patches. The score at each time-step, used to compute the reward for agent training, represents likelihood of object presence within the selection, determined by an object presence detector pre-trained using only image-level binary classification labels of object presence. Additionally, we propose a game termination condition that can be called by either side upon exhaustion of all ROI-containing patches, followed by the selection of a final patch from each. Upon termination, the agent is incentivised if ROI-containing patches are exhausted or disincentivised if an ROI-containing patch is found by the competitor. This competitive setup ensures minimisation of over- or under-segmentation, a common problem with WSS methods. Extensive experimentation across four datasets demonstrates significant performance improvements over recent state-of-the-art methods. Code: this https URL
https://arxiv.org/abs/2405.16628
https://arxiv.org/pdf/2405.16628.pdf