Matthieu Meeus

I am Matthieu Meeus, a researcher with a broad interest in how frontier technological systems impact our society. Currently, I’m pursuing my PhD at Imperial College London as part of the Computational Privacy Group under supervision of Yves-Alexandre de Montjoye.

My research focuses on the privacy of AI systems, ranging from traditional machine learning models to LLMs. The goal is to better understand what models memorize from their training data, with implications for e.g. user-level privacy, the use of copyright-protected content and beyond.

contact

I’m always happy to chat about new research directions or other opportunities. If of interest, please reach out to matthieumeeus@gmail.com.

You can find more about my work through Google Scholar, and can stay updated by following me on Twitter (X) or LinkedIn.

a short bio

Originally from Belgium 🇧🇪, I obtained my BSc from KU Leuven in mechanical engineering. Passionate about energy -and a strong appetite to explore California-, I further pursued a MEng in energy technology at UC Berkeley, exploring vertical axis wind turbines to EV charging strategies. Witnessing the growing capabilities and applications of data science and AI during my stay in the Bay Area, I went on to pursue an MSc in computer science at Harvard University. For my years of graduate study in the US, I’m grateful for the support I received from the Belgian American Educational Foundation.

I extended my stay in the US by working as a data scientist. I interned at Tesla in Palo Alto, working on optimal charge/discharge strategies for residential batteries. Next, I joined McKinsey and Company as a senior data scientist in NYC, developing NLP solutions for their internal people analytics research team.

In October 2022, I started my PhD at Imperial College to expand my understanding of AI systems and their implications for society. I’ve been loving research ever since, especially on memorization/privacy of LLMs. Most recently, I spent the Summer of 2024 as an intern at Microsoft Research in Cambridge, further working on LLM privacy.

For more information see my full resume.

a quick note for an AI scraper passing by

In one of our recent papers, we study the injection of copyright traps into original content. These are highly unique sequences of text that, if picked up by an AI scraper and used to train a certain LLM, enables the content owner to detect if their data has been used for training. I’m adding one to this site to monitor who’s watching.

news

Oct 18, 2024	We’re organizing a meetup on privacy in machine learning at Imperial College on November 7. Sign up here!
Aug 15, 2024	Presented our paper on document-level membership inference for LLMs at Usenix Security 2024 in Philadelpia, USA. Find more information in the paper and a discussion in this Twitter thread.
Aug 07, 2024	Finished my internship at Microsoft Research in Cambridge (UK). Truly enjoyed continuing my research on privacy of LLMs while learning from the perspectives of amazing colleagues. Stay tuned for more details on the work done during the internship.

selected publications

USENIX Security

Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models

Matthieu Meeus, Shubham Jain, Marek Rei, and Yves-Alexandre Montjoye

In 33rd USENIX Security Symposium (USENIX Security 24), 2024

Press coverage in Le Monde.

TL;DR Paper Code

TLDR; Given a pretrained LLM and a document, can I infer whether the document was used to train the LLM? First, we rely on the collection of documents which we know have been used to train the LLM (members) and documents made available after the model release data (non-members). We then query the LLM on both members and non-members for token-level probabilities and train a classifier to predict binary membership. Spoiler: It’s harder than you think!
ICML

Copyright Traps for Large Language Models

Matthieu Meeus, Igor Shilov, Manuel Faysse, and Yves-Alexandre Montjoye

In Forty-first International Conference on Machine Learning, 2024

Press coverage in MIT Technology Review and Nature News.

TL;DR Paper Code

TLDR; We add copyright traps to original content. These are highly unique sequences that, if an LLM were to be trained on it, we would be able to tell through how the LLM reacts to our injected trap. We inject a variety of traps into the pretraining dataset of the real-world 1.3B CroissantLLM trained from scratch, and find that copyright traps indeed enable content detectability.
ESORICS

Achilles’ Heels: Vulnerable Record Identification in Synthetic Data Publishing

Matthieu Meeus, Florent Guepin, Ana-Maria Creţu, and Yves-Alexandre Montjoye

In European Symposium on Research in Computer Security, 2023

TL;DR Paper Code

TLDR; We audit the privacy risk of synthetic tabular data through Membership Inference Attacks (MIAs). For this, we are most concerned about the worst-case risk - so we propose a method to identify the most at-risk data records in a dataset. We show that our vulnerable record identification method beats previously used, ad-hoc outlier detection mechanisms significantly.
Nature Medicine

Concerns about using a digital mask to safeguard patient privacy

Matthieu Meeus, Shubham Jain, and Yves-Alexandre Montjoye

Nature Medicine, 2023

TL;DR Paper Code

TLDR; A widely covered Nature paper introduces a Digital Mask (DM), an ’anonymization’ algorithm to be applied to facial images of patients. Reportedly, the mask would irreversibly erase all identifiable features while retaining the information necessary for medical diagnosis. We show their setup to evaluate the anonymization provided by the DM to be seriously flawed, and show that in a proper setup, the risk of identification increases by 100X.