Between data protection and research: New study puts two approaches to the test
Claims data are important for research on drug safety but contain sensitive patient information. To make them usable for research, data can be anonymized or synthetically generated. Both approaches are widely discussed, but have rarely been directly compared in a realistic setting. A research team from the group of Fabian Prasser at the Berlin Institute of Health at Charité (BIH), together with partners from the Health Data Lab (Forschungsdatenzentrum Gesundheit), the Institute for Applied Health Research Berlin (InGef), and MOSTLY AI, directly compared both approaches in a real-world medical research setting. In the interview, first author Mehmed Halilovic explains the key findings.
1. What question did you want to answer, what were you particularly interested in?
We took a previously published medication safety analysis and asked whether the main scientific conclusions could still be reproduced after anonymizing or synthesizing the underlying claims data. This helps to answer the question whether protected versions of claims data can still be used for meaningful medication safety research. What interested us especially was the direct comparison of anonymization and synthetization to see whether there are significant differences when applied in the exact same setting.
2. What specific challenges did you face when developing your approach?
One major challenge was the nature of the data itself. Health claims data can be small, complex, and high-dimensional, with rare but clinically important outcomes. In such datasets, even small changes introduced for privacy protection can affect the scientific results.
A second challenge was that privacy risks can change depending on the setting in which data is used, e.g. depending on what a potential attacker could realistically know. We therefore evaluated the data from different perspectives that made different assumptions about how trustworthy the recipients are or what further controls have been put in place.
In this process, we were able to leverage a lot of synergies with related research areas in our group that have led to the development of open-source tools for re-identification risk analysis and management. For example, the Risk Evaluation Assistant (REA; https://github.com/BIH-MI/risk-evaluation-assistant) supports the process of modelling different assumptions about data use contexts and the ARX Data Anonymization Tool (https://arx.deidentifier.org) allows to flexibly test different anonymization procedures. Without such tooling, this kind of privacy analysis would be much harder to perform in practice.
3. What surprised you and why? What was new?
The overall trade-off was not surprising, as stronger privacy protection usually reduces data utility. What was less obvious in advance was that our two methods lost utility in different ways.
Anonymizing data for use in untrusted contexts without further safeguards removed too much information to reproduce parts of the original analysis. Assuming further safeguards and trusted researchers gave a more balanced result. The data preserved substantially more analytical value, but the results carried noticeably more uncertainty than in the original analysis. Synthetic data, on the other hand, closely resembled the original data and looked similarly trustworthy at first glance, yet key numbers such as risk estimates shifted in ways that could change how a study is interpreted.
For us, this is an important message: A protected dataset can look useful, or even perform well on general fidelity measures, and still require careful validation for the actual scientific question. Protected data are therefore very valuable for exploration, collaboration, feasibility checks, and method development. Final conclusions should, however, still be validated against the original data.
4. What significance could this success have for future patients?
The benefit for patients is indirect, but important. Medication safety signals often lie in routine healthcare data that are difficult for researchers to access. Privacy-enhancing technologies, such as anonymization or synthetic data generation, can help to overcome this hurdle and support broader use of real-world data in areas such as public health, clinical and epidemiological research, and health services research. This means that routine healthcare data can be used more efficiently to study health outcomes, healthcare delivery and the safety and effectiveness of medical care.
5. Where do you see the translational bridge between research and application?
Data anonymization and synthetic data generation have seen substantial methodological development over several decades, yet only a few approaches have become part of routine medical research practice. Bridging this translational gap requires more than methodological innovation alone: it requires processes, tools and policies that make these technologies efficient, reliable and practical for use in large-scale data sharing infrastructures, such as those currently being established in Germany and across the European Union. Our work contributes to this by moving established methods closer to routine practice, implementing them in real-world settings, and evaluating what works, under which conditions, and why. This is closely linked to our research on tool support, including ARX and REA, which help translate methodological concepts into concrete, reproducible workflows for risk assessment and anonymization.