{"id":474,"date":"2021-07-30T18:44:35","date_gmt":"2021-07-30T16:44:35","guid":{"rendered":"https:\/\/www.wolter.tech\/?p=474"},"modified":"2021-10-06T14:05:18","modified_gmt":"2021-10-06T12:05:18","slug":"wavelet-packet-powered-deepfake-image-detection","status":"publish","type":"post","link":"https:\/\/www.wolter.tech\/?p=474","title":{"rendered":"Wavelet-Packet Powered Deepfake Image Detection"},"content":{"rendered":"\n<p>Modern neural networks generate realistic artificial images and audio. This development will allow us to create movies, music and audio effects never seen before. Yet at the same time, the new technology may enable new digital ways to lie.<\/p>\n\n\n\n<p>In response, the need for a diverse and reliable toolbox arises to identify artificial images and other content. This short blog post aims to summarize the main points regarding the use of the wavelet packet transform to identify artificially generated deepfake images. The key observation is that wavelet packet coefficients are distributed differently for real and fake images.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"429\" src=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization-1024x429.png\" alt=\"\" class=\"wp-image-489\" srcset=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization-1024x429.png 1024w, https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization-300x126.png 300w, https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization-768x321.png 768w, https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/10\/packet_visualization.png 1314w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure>\n\n\n\n<p>The image above illustrates this. The leftmost column shows a single real image from the Flickr-Faces-HQ data set as well as an artificially generated image for reference. To study the feasibility of wavelet packets for deepfake detection third-degree Haar-Wavelet packet coefficients are computed for 5k real and fake images using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/v0lta\/PyTorch-Wavelet-Toolbox\" target=\"_blank\">PyTorch-Wavelet-Toolbox<\/a>. Comparing the mean coefficients in the center as well as their standard distribution, we notice differences especially as the frequency increases along the diagonal. The standard deviation is significantly different in the background parts of the images across the board. The differences suggest a possibility to separate real from fake based on the wavelet packet coefficients.<\/p>\n\n\n\n<p>A first experiment explores the separability of images from the Flicker-Faces-HQ dataset as well as<a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/NVlabs\/stylegan\" target=\"_blank\"> style-gan<\/a> generated images. Working with 63k 128 by 128 images from each source the task is to identify the origin of an image.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full\"><a href=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/07\/train_deepfake_classifier.png\"><img loading=\"lazy\" decoding=\"async\" width=\"321\" height=\"282\" src=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/07\/train_deepfake_classifier.png\" alt=\"\" class=\"wp-image-480\" srcset=\"https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/07\/train_deepfake_classifier.png 321w, https:\/\/www.wolter.tech\/wordpress\/wp-content\/uploads\/2021\/07\/train_deepfake_classifier-300x264.png 300w\" sizes=\"auto, (max-width: 321px) 100vw, 321px\" \/><\/a><\/figure><\/div>\n\n\n\n<p>The plot above shows the convergence of a classifier trained to identify the source of an image. The wavelet packets allow the classifier to converge faster with performance improvements during all stages of the training.<\/p>\n\n\n\n<p>If you would like to find out more the s<a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/gan-police\/frequency-forensics\" target=\"_blank\">ource code<\/a> as well as a <a rel=\"noreferrer noopener\" href=\"https:\/\/arxiv.org\/pdf\/2106.09369.pdf\" target=\"_blank\">preprint<\/a> are now freely available online.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Modern neural networks generate realistic artificial images and audio. This development will allow us to create movies, music and audio effects never seen before. Yet at the same time, the new technology may enable new digital ways to lie. In response, the need for a diverse and reliable toolbox arises to identify artificial images and &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.wolter.tech\/?p=474\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Wavelet-Packet Powered Deepfake Image Detection&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[25,12,28],"class_list":["post-474","post","type-post","status-publish","format-standard","hentry","category-research-projects","tag-frequency-domain","tag-machine-learning","tag-wavelets","entry"],"_links":{"self":[{"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/posts\/474","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=474"}],"version-history":[{"count":7,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/posts\/474\/revisions"}],"predecessor-version":[{"id":491,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=\/wp\/v2\/posts\/474\/revisions\/491"}],"wp:attachment":[{"href":"https:\/\/www.wolter.tech\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=474"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=474"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wolter.tech\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=474"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}