<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Opinion | MeVer</title>
    <link>https://mever.gr/tag/opinion/</link>
      <atom:link href="https://mever.gr/tag/opinion/index.xml" rel="self" type="application/rss+xml" />
    <description>Opinion</description>
    <generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Fri, 18 Dec 2020 00:00:00 +0000</lastBuildDate>
    <image>
      <url>https://mever.gr/media/logo_hu_ecbb58bff06221f9.png</url>
      <title>Opinion</title>
      <link>https://mever.gr/tag/opinion/</link>
    </image>
    
    <item>
      <title>Deepfakes: An Emerging Internet Threat and their Detection</title>
      <link>https://mever.gr/post/deepfakes-an-emerging-internet-threat-and-their-detection/</link>
      <pubDate>Fri, 18 Dec 2020 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/deepfakes-an-emerging-internet-threat-and-their-detection/</guid>
      <description>&lt;p&gt;Towards the end of 2020, I had the opportunity to give a talk on Deepfakes in the 60th &lt;a href=&#34;https://www.ai4eu.eu/ai4eu-webcafes&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AI4EU WebCafe&lt;/a&gt;. With deepfakes being already with us for more than two years now, we have already witnessed several real-world cases of deepfakes with both positive and negative impact, while their use has recently been commoditized by popular apps such as ZAO and Reface. At the same time, it seems that there is an exponentially growing interest in the area from the academic community as indicated by the quickly growing number of papers related to the topic.&lt;/p&gt;
&lt;p&gt;Even though convincing deepfake generation is still hard and resource-intensive (both in terms of computing power and manual video editing skills), there has been immense progress in the area and numerous approaches and techniques that have led to remarkable improvements in the quality and realism of the resulting deepfakes, as well as on the requirements in terms of number of training samples and compute power. The following image shared by Ian Goodfellow, the father of Generative Adversarial Networks (GANs) vividly illustrates the progress achieved during the last years.&lt;/p&gt;
&lt;p&gt;TWEET&lt;/p&gt;
&lt;p&gt;What is more, there is now a variety of open-source tools and repositories, with &lt;a href=&#34;https://faceswap.dev/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FaceSwap&lt;/a&gt; and &lt;a href=&#34;https://github.com/iperov/DeepFaceLab&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFaceLab&lt;/a&gt; being the most popular ones, that have considerably lowered the barrier to entry even to non-experts (but still a bit tech-savvy).&lt;/p&gt;
&lt;p&gt;Following the progress of deepfake generation approaches, there has been commensurate progress in the inverse problem of deepfake detection. There have been numerous types of approaches tackling the problem. A first class of such approaches try to spot deepfake artifacts, much like a careful human inspector would, for instance, fuzzy and blurry areas around the lips, earlobes and hair, lack of symmetries, e.g. different colors between the left and right eye, fuzzy background, etc. There are also approaches that attempt to extract physiological signals from a video in question, e.g. eye blinking or human pulse, and detect whether there are inconsistencies or unrealistic patterns in their evolution. Other approaches adopt a more general supervised learning paradigm, trying to learn distinctive deepfake patterns by looking into sets of both deepfake and authentic images. Last but not least, as in the case of media forensics approaches, there are deepfake detection methods that try to uncover deepfake artifacts in the frequency domain.&lt;/p&gt;
&lt;p&gt;Even though the area is quite young, there are already a few very good surveys on the topic:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2001.06564&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Media Forensics and DeepFakes: an overview&lt;/a&gt; by Verdoliva&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2001.00179&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection&lt;/a&gt; by Tolosana et al.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2004.11138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The Creation and Detection of Deepfakes: A Survey&lt;/a&gt; by Misky and Lee&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Another great landmark for deepfake detection research has been the launch and completion of the &lt;a href=&#34;https://www.kaggle.com/c/deepfake-detection-challenge&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFake Detection Challenge&lt;/a&gt;. The challenge attracted more than 2100 participants and offered a very hard benchmarking setting, which revealed the main weakness of state-of-the-art deepfake detection methods, i.e. their limited ability to generalize to new types and forms of deepfake content. Yet, at the same time, the challenge pushed the performance boundary of solutions and raised awareness about new detection approaches to the community. Our MeVer team also participated in the challenge in the context of the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project and was quite successful, ranked among the top 5% of submitted solutions. We had already provided an overview of our solution in a &lt;a href=&#34;https://mever.gr/2020/06/16/mever-participation-in-deepfake-detection-challenge/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;previous blog post&lt;/a&gt;. Since then, we have also performed a number of improvements after releasing an alpha version of our service to our partners in the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project. You can get a glimpse on how our latest deepfake detection service looks in our latest &lt;a href=&#34;https://mever.gr/2020/12/17/mever-tools-for-image-and-video-verification-within-weverify/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blog post about verification tools&lt;/a&gt;, where we present all tools we developed so far in WeVerify.&lt;/p&gt;
&lt;p&gt;Closing, my thoughts on the current state of deepfakes are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Deepfake generation still requires a certain degree of expertise and the resulting deepfakes are in most cases still possible to detect by a trained human observer, but this is likely to change in the near future (2-3 years), making possible the creation of highly realistic deepfakes with very little manual intervention and computational resources.&lt;/li&gt;
&lt;li&gt;There are a variety of deepfake detection approaches, each working well on different kinds of deepfakes. A common issue among all of them is the limited generalization ability to cases that were not well represented in the training set. However, there are certain approaches that claim to be general (e.g. those based on analysis of physiological signals and analysis on the frequency domain), at least given the current state of deepfake generation technology.&lt;/li&gt;
&lt;li&gt;Currently, the negative impact of deepfakes has mainly affected individuals, e.g. revenge porn, cyberbullying, while there have also been cases where there was broader socio-political impact as a result of deepfakes or even the suspicion of deepfakes (e.g. Gabon president video).&lt;/li&gt;
&lt;li&gt;We shouldn’t be fooled by the fact that there has been limited use of deepfakes in the context of disinformation campaigns to date. This is likely not due to the limitations of the technology but probably due to the fact that there are already established disinformation techniques that still work effectively and do not require any technical expertise. In the near future, however, this could change, so the risk of deepfakes for disinformation should not be underestimated.
The video recording of the talk and the accompanying slides are available below.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;p&gt;SLIDES&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>On the Ephemerality of Web Media</title>
      <link>https://mever.gr/post/on-the-ephemerality-of-web-media/</link>
      <pubDate>Tue, 10 Mar 2020 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/on-the-ephemerality-of-web-media/</guid>
      <description>&lt;p&gt;A lot of our research and development activities rely on large collections of web media content sourced from social media platforms, such as YouTube and Twitter, and then manually curated and annotated by our researchers with the purpose of creating “ground truth” datasets. This helps us train machine learning models on specific tasks and then benchmark those models along with competing approaches in order to select the best method per case.
It goes without saying that we spend loads of effort on developing scripts for crawling, monitoring, extracting and fetching the necessary data and content related to the target task, and then even more effort on curating, cleaning and labeling (aka annotating) the collected datasets. Especially, content labelling is particularly challenging due to the subjective nature of the task, e.g. different people may perceive the same content as belonging to different categories, while there are additional issues in specific annotation tasks, e.g. when dealing with NSFW and disturbing content.&lt;/p&gt;
&lt;p&gt;Putting all the above issues aside, a major challenge that we, scientists, face when dealing with data and content that has been sourced from online sources is its ephemeral nature. Online data and information may cease to be available at its source. For instance, YouTube users may opt to delete one or more of their previously uploaded videos &lt;a href=&#34;https://www.quora.com/Why-do-YouTubers-delete-some-of-their-videos-even-when-they%E2%80%99re-clearly-good&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;for a variety of reasons&lt;/a&gt;. An even more common case is that the media platforms decide to take down content due to violation of their terms of service or copyright infringement. Such a case happened in April 2018 when &lt;a href=&#34;https://www.reuters.com/article/us-alphabet-youtube/youtube-deletes-5-million-videos-for-content-violation-idUSKBN1HV29H&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;YouTube removed 5 million videos on the basis of content violation&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;To demonstrate the issue we face, we would like to share our experience and concerns regarding two datasets that we recently (2019) created to support &lt;a href=&#34;https://records.sigmm.org/2019/10/17/dataset-column-datasets-for-online-multimedia-verification/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;research on the problem of online media verification&lt;/a&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;a href=&#34;https://mklab.iti.gr/results/fake-video-corpus/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Fake video corpus (FVC)&lt;/a&gt; [1], a dataset of verified and debunked user-generated videos from YouTube, Facebook and Twitter. The FVC dataset contained 200 fake video cases and 180 real video cases. Following a semi-automatic procedure, as described in [1], we collected 3,262 near duplicates of the above fake videos and 1,933 of the above real videos. By the time it was publicly released, February 2019, in total 5,575 videos from YouTube, Facebook and Twitter were available online.&lt;/li&gt;
&lt;li&gt;The &lt;a href=&#34;https://ndd.iti.gr/fivr/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FIVR-200K&lt;/a&gt; [2] dataset was collected to simulate the problem of Fine-grained Incident Video Retrieval (FIVR). It offers a single means of evaluating several video retrieval tasks as special cases. The FIVR-200K, which was collected in March 2018, consisted of 225,960 YouTube videos. The videos were collected from January 1st 2013 to December 31st 2017 following the procedure described in [2].
During our work with these datasets, we regularly noticed that specific content items were not available online. To investigate this in a more systematic way, in February 2020, we looked into both datasets in terms of video availability. We concluded that a significant number of the videos that were initially included in the datasets were not unavailable anymore.
With respect to the FVC, the number of videos that have been removed by the three video platforms amounts to a 21,5% reduction of the number of videos; specifically, we measured a 18,4% reduction in the number of the initial video cases and 21,8% of the video duplicates.
Similarly, from the 225,960 YouTube videos of the FIVR-200K dataset, only 190,528 were still available on YouTube in February 2020, meaning that more than 30,000 videos of the dataset had been removed. This corresponds to a reduction of 15,7% in the total amount of videos.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;TABLE&lt;/p&gt;
&lt;p&gt;Summary of the number of available and unavailable videos of the FVC and the FIVR-200K at the time they were created and now (February 2020).&lt;/p&gt;
&lt;p&gt;For that reason, the unavailable videos had to be removed from the corresponding dataset releases. The issue of online content ephemerality concerns the research community since the effort to create such a dataset is large and the reproducibility of the corresponding experiments is really harmed. Potential solutions to the issue that we see adopted by the research community include the following among others:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Release extracted features from the original media collection: This is a practice that is common in the computer vision and multimedia community. Given the prevalence of standard and widely used feature extractors, both manually crafted (SIFT, SURF) and neural network based (VGG, ResNet, Xception, etc.), it is still possible to perform a variety of interesting experiments when one has access to the extracted features and not the original content. On the downside, maintaining and distributing a variety of features from massive media collections is expensive in terms of storage and bandwidth, while the rapid development of new feature extractors is expected to soon make such releases obsolete.&lt;/li&gt;
&lt;li&gt;Release reduced versions of the original media collection: This involves the periodic updating of the original collection with the purpose of removing any media items that are not available online anymore. In cases where a sizeable portion of the original collection remains available, this is an acceptable approach. However, as became clear from our above discussion, large reductions (in the order of 20%) are very likely soon (one year) after a dataset is released. This is especially problematic when the original dataset contains low-frequency classes that are of interest to the target task.&lt;/li&gt;
&lt;li&gt;Retaining and privately sharing of the original collection: Even though this practice is in clear violation of most platforms’ Terms of Service and pertinent regulation (including Copyright Law and GDPR), we regularly see this happen among members of the research community. This is because it is the only approach that ensures full reproducibility of research based on the original collection. The fact that this practice is still common despite its lack of legal basis may be telltale of the researchers’ agony to ensure that their research remains reproducible and relevant in the long run.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;It becomes clear that this is a challenge that has no fully satisfactory solution. For that purpose, we expect that ongoing and future research on social media and the Web will continue to struggle around this problem, but we are hopeful that a set of best practices will gradually emerge that will strike a good compromise between reproducibility, research value and legal compliance.&lt;/p&gt;
&lt;p&gt;[1] Papadopoulou, O., Zampoglou, M., Papadopoulos, S. &amp;amp; Kompatsiaris, I. (2019), &lt;a href=&#34;https://www.emerald.com/insight/content/doi/10.1108/OIR-03-2018-0101/full/html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;“A corpus of debunked and verified user-generated videos”&lt;/a&gt;, Online Information Review, Vol. 43 No. 1, pp. 72-88.
[2] Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., &amp;amp; Kompatsiaris, I. (2019), &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/8668422&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;“FIVR: Fine-grained Incident Video Retrieval”&lt;/a&gt;. IEEE Transactions on Multimedia 21(10), 2638-2652.&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>
