NeurIPS 2024 Competition
Erasing the Invisible:
A Stress-Test Challenge for Image Watermarks

Competition 101

The logistics of this competition are very simple.

• Input: A folder of images with invisible watermarks. In the Black-box Track, no additional information is provided. In the Beige-box Track, details about the watermarking techniques used are also included.
• Output: Your task is to develop attacks to remove the watermarks while preserving the quality of the images. Submissions will be in the form of a folder of images after attacks.
• Evaluation: Submissions will be assessed based on their effectiveness of the watermark removal and the preservation of image quality, resulting in a final composite score.

We encourage participants to read WAVES paper to find more details regarding the watermarking algorithms and their evaluation. In this paper, you can also find baseline attacks.

Warm-up Phase (Finished)

• Warm-up Kit: Github link

This is a preparation phase. No submissions are required. The warm-up kit provides example watermarked images and various metrics for evaluating attack potency and image quality. It will help you become familiar with the competition pipeline and prepare your attacks.

Note: The models, watermarks, images provided in the Warm-up Kit are for practice only. The submission phase will use different ones. The final evaluation metric in the submission phase will be based on the metrics provided in the Warm-up Kit.

Submission Phase (Ongoing)

This is the primary competition phase. The competition is hosted on Codabench. There are two tracks (see Tracks for details), and you can participate in both. Please follow the guidelines and submit your results on Codabench. Here are the two tracks on Codabench:

• Black Box Track: Codabench link
• Beige Box Track: Codabench link