</code></pre></div></div> <h2 id="use-cases">Use Cases</h2> <p>Synthetic data has two primary use cases:</p> <ol> <li>Descriptive analytics (histograms, heatmaps, column correlations, basic statistics like counting, averages, standard deviations, and so on).</li> <li>Machine learning (building models, extending datasets, etc.)</li> </ol> <p>While <strong>SynDiffix</strong> serves both use cases, it is especially good at descriptive analytics. The quality of descriptive analytics is many times that of other synthetic data products.</p> <h2 id="usage">Usage</h2> <p>Obtaining this accuracy improvement, however, requires a different usage style compared to other products. The intended usage style of other products is “<em>one size fits all</em>”: a single monolithic synthetic dataset serves all use cases. It’s like expecting a vehicle to be good at city maneuverability, highway cruising, and hauling lumber.</p> <p>By contrast, with <strong>SynDiffix</strong>, a different <em>tailored</em> synthetic dataset can be produced for each use case: like having a Smart car for the city, a Mercedes S-class for the highway, and a Ford F-150 for hauling stuff.</p> <p><img src="/assets/img/usage.png" alt="SynDiffix usage style" class="img-fluid mx-auto d-block" style="max-width: 550px; width: 100%;"></p> <p>For instance, suppose the analyst is interested in a heatmap with columns C and E. With other synthetic data products, one would synthesize the complete table, and then make the heatmap with only columns C and E. With <strong>SynDiffix</strong>, one would create a synthetic table consisting of only those two columns and obtain much better results.</p> <p><strong>SynDiffix</strong> has anonymization mechanisms that allow for literally thousands of column combinations without compromising anonymity. This is not the case with other products, where anonymity is weakened with each new data synthesis.</p> <h2 id="accuracy">Accuracy</h2> <p>Here are three scatter plots showing the synthetic and real points for a 2-column dataset for <strong>SynDiffix</strong>, the commercial product MostlyAI, and the open-source implementation of CTGAN by SDV.</p> <p><img src="/assets/img/scatter.png" alt="SynDiffix accuracy" class="img-fluid mx-auto d-block" style="max-width: 600px; width: 100%;"></p> <p>The black dots are the original data and the blue dots are the synthetic overlaid on the original data. SynDiffix is far more accurate. More examples can be found in the <a href="https://arxiv.org/abs/2311.09628" rel="external nofollow noopener" target="_blank">arXiv paper</a>.</p> <h2 id="anonymity">Anonymity</h2> <p>We measured the anonymity of <strong>SynDiffix</strong> and a number of other products using the <a href="https://github.com/statice/anonymeter" rel="external nofollow noopener" target="_blank">Anonymeter</a> tool developed by Statice. The following figure shows the results of measuring the effectiveness of 100s of attacks over multiple different tables (again, see the <a href="https://arxiv.org/abs/2311.09628" rel="external nofollow noopener" target="_blank">arXiv paper</a> for details). Any score below 0.5 can be regarded as having strong anonymity, and scores below 0.2 are very strong. The “noAnon” plot measures attacks against the original data, and is there for calibration.</p> <p><img src="/assets/img/privacy.png" alt="SynDiffix privacy" class="img-fluid mx-auto d-block" style="max-width: 400px; width: 100%;"></p> <p>As these results show, <strong>SynDiffix</strong> and most of the other products have very strong anonymization with respect to the attack used by Anonymeter.</p> </article> </div> </div> <footer class="sticky-bottom mt-5"> <div class="container"> © Copyright 2023 Open Diffix. Powered by <a href="https://jekyllrb.com/" target="_blank" rel="external nofollow noopener">Jekyll</a> with <a href="https://github.com/alshedivat/al-folio" rel="external nofollow noopener" target="_blank">al-folio</a> theme. Hosted by <a href="https://pages.github.com/" target="_blank" rel="external nofollow noopener">GitHub Pages</a>. </div> </footer> <script src="https://cdn.jsdelivr.net/npm/jquery@3.6.0/dist/jquery.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script> <script src="/assets/js/bootstrap.bundle.min.js"></script> <script src="https://cdn.jsdelivr.net/npm/mdbootstrap@4.20.0/js/mdb.min.js" integrity="sha256-NdbiivsvWt7VYCt6hYNT3h/th9vSTL4EDWeGs5SN3DA=" crossorigin="anonymous"></script> <script defer src="https://cdn.jsdelivr.net/npm/medium-zoom@1.0.8/dist/medium-zoom.min.js" integrity="sha256-7PhEpEWEW0XXQ0k6kQrPKwuoIomz8R8IYyuU1Qew4P8=" crossorigin="anonymous"></script> <script defer src="/assets/js/zoom.js?7b30caa5023af4af8408a472dc4e1ebb"></script> <script defer src="https://unpkg.com/bootstrap-table@1.22.1/dist/bootstrap-table.min.js"></script> <script src="/assets/js/no_defer.js?d633890033921b33e0ceb13d22340a9c"></script> <script defer src="/assets/js/common.js?acdb9690d7641b2f8d40529018c71a01"></script> <script defer src="/assets/js/copy_code.js?c9d9dd48933de3831b3ee5ec9c209cac" type="text/javascript"></script> <script type="text/javascript">window.MathJax={tex:{tags:"ams"}};</script> <script defer type="text/javascript" id="MathJax-script" src="https://cdn.jsdelivr.net/npm/mathjax@3.2.0/es5/tex-mml-chtml.js"></script> <script defer src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script> <script type="text/javascript">function progressBarSetup(){"max"in document.createElement("progress")?(initializeProgressElement(),$(document).on("scroll",function(){progressBar.attr({value:getCurrentScrollPosition()})}),$(window).on("resize",initializeProgressElement)):(resizeProgressBar(),$(document).on("scroll",resizeProgressBar),$(window).on("resize",resizeProgressBar))}function getCurrentScrollPosition(){return $(window).scrollTop()}function initializeProgressElement(){let e=$("#navbar").outerHeight(!0);$("body").css({"padding-top":e}),$("progress-container").css({"padding-top":e}),progressBar.css({top:e}),progressBar.attr({max:getDistanceToScroll(),value:getCurrentScrollPosition()})}function getDistanceToScroll(){return $(document).height()-$(window).height()}function resizeProgressBar(){progressBar.css({width:getWidthPercentage()+"%"})}function getWidthPercentage(){return getCurrentScrollPosition()/getDistanceToScroll()*100}const progressBar=$("#progress");window.onload=function(){setTimeout(progressBarSetup,50)};</script> </body> </html>
0 commit comments