<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>News | MeVer</title>
    <link>https://mever.gr/tag/news/</link>
      <atom:link href="https://mever.gr/tag/news/index.xml" rel="self" type="application/rss+xml" />
    <description>News</description>
    <generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Tue, 10 Dec 2024 00:00:00 +0000</lastBuildDate>
    <image>
      <url>https://mever.gr/media/logo_hu_ecbb58bff06221f9.png</url>
      <title>News</title>
      <link>https://mever.gr/tag/news/</link>
    </image>
    
    <item>
      <title>Reducing Inference Energy Consumption Using Dual Complementary CNNs</title>
      <link>https://mever.gr/post/reducing-inference-energy-consumption-using-dual-complementary-cnns/</link>
      <pubDate>Tue, 10 Dec 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/reducing-inference-energy-consumption-using-dual-complementary-cnns/</guid>
      <description>&lt;p&gt;In this post we present our dual CNN methodology, that reduces inference energy requirements of CNNs in image classification tasks on low-resource devices. Further details about our approach and associated experiments are included  in our paper &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S0167739X24005703?dgcid=author&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Reducing inference energy consumption using dual complementary CNNs&lt;/a&gt;, which has been accepted for publication in the journal of Future Generation Computer Systems.&lt;/p&gt;
&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;Deep neural networks (DNNs) play a critical role in solving complex problems in smart environments, with growing applications in IoT, smart homes, and edge computing. However, deploying large DNNs on edge devices poses significant challenges due to their high energy demands, which can quickly deplete battery life and shorten device energy autonomy. Existing DNN compression techniques aim to address these issues by creating smaller models with fewer parameters and reduced computational requirements. While these methods enhance efficiency, they often come with trade-offs, including potential loss of accuracy and significant computational costs for re-training and fine-tuning. To address these limitations we propose a methodology that uses two small complementary CNNs, that combined they increased their predictive performance while still keeping energy consumption low.&lt;/p&gt;
&lt;h2 id=&#34;dual-complementary-cnn-architecture&#34;&gt;Dual complementary CNN architecture&lt;/h2&gt;
&lt;p&gt;Our architecture consists of a pair of small convolutional neural networks (CNNs) that exhibit a relatively high complementarity score from a given pool of pretrained models. The selection process ensures that the two selected CNN models have the smallest size possible while together covering most of the prediction cases in a given dataset. This setup is further enhanced by the use of a caching mechanism that remembers previous similar classifications and skips the CNN inference if the same input is detected, leading to further energy reduction.
For each input, the process begins with the first CNN generating a prediction and calculating a confidence score derived from its logits vector. An output logits vector of an artificial neural network represents the raw, unnormalized scores from the network&amp;rsquo;s final layer, where each value corresponds to a class in the dataset and indicates the relative likelihood of the input belonging to that class. This confidence score is then evaluated against a predefined threshold, which determines whether the second CNN needs to be activated. If the score falls below the threshold, the second CNN is invoked to refine the prediction. When both CNNs are utilized, their confidence scores are compared, and the final prediction is selected based on the higher confidence level. Additionally, with the memory component for every input, a unique fingerprint is generated and used as a key to access a hash table. If the hash table contains a stored classification for the fingerprint, the cached result is returned immediately, bypassing the need to invoke the CNNs leading to further energy reduction.&lt;/p&gt;
&lt;h2 id=&#34;the-concept-of-complementarity&#34;&gt;The Concept of Complementarity&lt;/h2&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/dual_cnn_complementarity-idea-demonstration.png&#34; alt=&#34;complementarity_idea&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Complementarity based on the predictions of two CNN models.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The concept of complementarity between two CNN models is illustrated using the above figure where the dataset is represented by a gray box, and the models’ correct predictions are depicted as blue and green circles. Complementarity is defined as the symmetric difference between the correct predictions of the two models, emphasizing their ability to complement each other&amp;rsquo;s strengths. Four scenarios highlight how complementarity impacts prediction performance and resource efficiency.&lt;/p&gt;
&lt;p&gt;In the first scenario (a), both models correctly predict much of the dataset but overlap significantly, leading to low complementarity, redundancy, and higher energy consumption. The second scenario (b) shows high complementarity and extensive prediction coverage, but the larger model&amp;rsquo;s energy demands make the configuration inefficient. The third example (c) illustrates perfect complementarity with no overlap but limited prediction coverage, resulting in suboptimal accuracy. The fourth scenario (d) demonstrates the ideal balance, where models of similar size achieve high complementarity and cover most of the dataset with minimal overlap, optimizing both performance and resource utilization.&lt;/p&gt;
&lt;h2 id=&#34;results&#34;&gt;Results&lt;/h2&gt;
&lt;p&gt;We conducted our experiments on four datasets: &lt;a href=&#34;https://www.cs.toronto.edu/~kriz/cifar.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CIFAR-10&lt;/a&gt;, &lt;a href=&#34;https://image-net.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ImageNet&lt;/a&gt;, &lt;a href=&#34;https://www.kaggle.com/datasets/puneet6060/intel-image-classification&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intel Image Classification&lt;/a&gt;, and &lt;a href=&#34;https://www.kaggle.com/datasets/zalando-research/fashionmnist&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FashionMNIST&lt;/a&gt;. For each dataset, our architecture setups (CI3, IM3, IN3, and FM3, respectively) demonstrated reduced energy consumption while maintaining high accuracy. In contrast, other model compression techniques either achieved better performance at the cost of significantly higher energy consumption or reduced energy usage with substantial drops in accuracy. For more details on the other comparison methods you can read the paper &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S0167739X24005703?dgcid=author&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/dual_cnn_efficiency_quad.png&#34; alt=&#34;efficiency&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Current and Energy consumption measurements of tested configurations for (a) CIFAR-10, (b) ImageNet, (c) Intel and (d) FashionMNIST datasets.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/dual_cnn_performance_quad.png&#34; alt=&#34;performance&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Performance of tested configurations for (a) CIFAR-10, (b) ImageNet, (c) Intel and (d) FashionMNIST datasets.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;When utilizing the optional memory component, we observe significantly lower energy consumption in environments with repeated similar inputs. In the following figure, our implementations without the memory component are represented as CI3, IM3, IN3, and FM3, while those with the memory component (using two different methods) are labeled as CI4 and CI5, IM4 and IM5, IN4 and IN5, and FM4 and FM5. Notably, increasing the number of duplicated samples results in little to no additional energy consumption.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/dual_cnn_memory_quad.png&#34; alt=&#34;memory&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Energy consumption comparisons using memory component with duplicated samples.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;We also make available the code to reproduce our experiments on &lt;a href=&#34;https://github.com/michaelkinnas/Reducing-Inference-Energy-Consumption-Using-Two-Complementary-CNNs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this&lt;/a&gt; GitHub repository.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Leveraging Representations from Intermediate Encoder-Blocks for Synthetic Image Detection</title>
      <link>https://mever.gr/post/representations-from-intermediate-encoder-blocks/</link>
      <pubDate>Wed, 10 Jul 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/representations-from-intermediate-encoder-blocks/</guid>
      <description>&lt;p&gt;In this post we present the essential parts of our method RINE, described in the paper &lt;em&gt;&lt;a href=&#34;https://arxiv.org/abs/2402.19091&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Leveraging Representations from Intermediate Encoder-Blocks for Synthetic Image Detection&lt;/a&gt;&lt;/em&gt;, which has been accepted by the &lt;a href=&#34;https://eccv2024.ecva.net/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;European Conference on Computer Vision (ECCV 2024)&lt;/a&gt;.&lt;/p&gt;
&lt;h1 id=&#34;motivation&#34;&gt;Motivation&lt;/h1&gt;
&lt;p&gt;Recent research on Synthetic Image Detection (SID) has led to strong evidence on the advantages of representations extracted by foundation models, exhibiting exceptional generalization capabilities on GAN and Diffusion model generated data &lt;a href=&#34;https://arxiv.org/abs/2302.10174&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;(Ojha et al. 2023)&lt;/a&gt;. Motivated by this success, we hypothesize that further performance gains are possible by leveraging representations from intermediate layers, which carry low-level visual information, in addition to representations from the final layer that primarily carry high-level semantic information.&lt;/p&gt;
&lt;h1 id=&#34;method&#34;&gt;Method&lt;/h1&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/rine-architecture.png&#34; alt=&#34;architecture&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;The RINE architecture. A batch of images is processed by CLIP&amp;rsquo;s image encoder. The concatenation of intermediate CLS tokens is first projected and then multiplied with the blocks&amp;rsquo; scores, estimated by the Trainable Importance Estimator (TIE) module. Summation across the second dimension results in one feature vector per image. Finally, after the second projection and the consequent classification head modules, two loss functions are computed. Binary cross-entropy directly optimizes SID, while the contrastive loss assists the training by forming a dense feature vector cluster per class. Figure by Christos Koutlis&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;feature-extraction&#34;&gt;Feature Extraction&lt;/h2&gt;
&lt;p&gt;After the standard image pre-processing (patch extraction and projection, positional encoding), CLIP&amp;rsquo;s Visual Transformer processes the input tokens $\mathbf{Z}_0\in\mathbb{R}^{b\times (p+1)\times d}$ as below:&lt;/p&gt;
&lt;p&gt;$$
\tilde{\mathbf{Z}_l}=\text{MSA}(\text{LN}(\mathbf{Z}_{l-1}))+\mathbf{Z}_{l-1}
$$&lt;/p&gt;
&lt;p&gt;$$
\mathbf{Z}_l=\text{MLP}(\text{LN}(\tilde{\mathbf{Z}_l}))+\tilde{\mathbf{Z}_l}
$$&lt;/p&gt;
&lt;p&gt;with $l$ denoting the block&amp;rsquo;s index.&lt;/p&gt;
&lt;p&gt;Then, we define the Representations from Intermediate Encoder-blocks (RINE) $\mathbf{K}$, as the concatenation of &lt;code&gt;CLS&lt;/code&gt; tokens from the corresponding $n$ blocks:&lt;/p&gt;
&lt;p&gt;$$
\mathbf{K}=\oplus\lbrace\mathbf{Z}_l^{\text{[0]}}\rbrace_{l=1}^n\in\mathbb{R}^{b\times n\times d}
$$&lt;/p&gt;
&lt;p&gt;where $\oplus$ denotes concatenation, and $\mathbf{Z}_l^{\text{[0]}}\in\mathbb{R}^{b\times 1\times d}$ denotes the &lt;code&gt;CLS&lt;/code&gt; token from the output of block $l$. We keep CLIP, which produces these features, frozen during training and use $\mathbf{K}$ to construct discriminative features for the SID task.&lt;/p&gt;
&lt;h2 id=&#34;learning&#34;&gt;Learning&lt;/h2&gt;
&lt;p&gt;A projection network $\mathfrak{Q}_1$ with linear mappings, &lt;code&gt;ReLU&lt;/code&gt; activations, and &lt;code&gt;Dropout&lt;/code&gt; regulatizations, processes the input representations $\mathbf{K}$. Its output is then adjusted, with the use of a Trainable Importance Estimator (TIE) module, in order to reflect the importance of each Transformer block to the SID task:&lt;/p&gt;
&lt;p&gt;$$
\tilde{\mathbf{K}}^{(ik)}=\sum_l^n\mathcal{S}(\mathbf{A})^{(lk)}\cdot\mathbf{K}_q^{(ilk)}
$$&lt;/p&gt;
&lt;p&gt;Another projection network $\mathfrak{Q}_2$ is then applied before the classification head that estimates the probability $\hat{y}_i$ of image $i$ to be fake.&lt;/p&gt;
&lt;p&gt;Finally, we use a combination of binary cross-entropy and supervised contrastive loss to optimize our model:&lt;/p&gt;
&lt;p&gt;$$
\mathfrak{L}_{CE}=-\sum_{i=1}^by_i\text{log}\hat{y}_i+(1-y_i)\cdot\text{log}(1-\hat{y}_i)
$$&lt;/p&gt;
&lt;p&gt;$$
\mathfrak{L}_{Cont.}=-\sum_{i=1}^b\frac{1}{G(i)}\sum_{g\in G(i)}\text{log}\frac{\text{exp}(\mathbf{z}_i\cdot \mathbf{z}_g/\tau)}{\sum_{a\in A(i)}\text{exp}(\mathbf{z}_i\cdot \mathbf{z}_a/\tau)}
$$&lt;/p&gt;
&lt;p&gt;$$
\mathfrak{L}=\mathfrak{L}_{CE}+\xi\cdot\mathfrak{L}_{Cont.}
$$&lt;/p&gt;
&lt;h1 id=&#34;results&#34;&gt;Results&lt;/h1&gt;
&lt;p&gt;In the below Table, we present the performance of our method versus the competing ones in terms of accuracy. Our 1-class model outperforms all state-of-the-art methods irrespective of training class number. On average, we surpass the state-of-the-art by +9.4% with the 1-class model, by +6.8% with the 2-class model, and by +10.6% with the 4-class model. We obtain the best score in 14 out of 20 test datasets, and simultaneously the first and second best performance in 10 of them. The biggest performance gain is on the SAN dataset (+16.9%).&lt;/p&gt;
&lt;p&gt;Additionally, in the &lt;a href=&#34;https://arxiv.org/abs/2402.19091&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper&lt;/a&gt; you can find ablations that support our architectural choices, further experiments regarding robustness to perturbations, the effects of training duration, and training set size, the fact that considering more ProGAN object classes during training does not reduce generalization, and even more&amp;hellip;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/rine-results-acc.png&#34; alt=&#34;accuracy&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Accuracy (ACC) scores of baselines and our model across 20 test datasets. The second column (# cl.) presents the number of used training classes. Best performance is denoted with bold and second to best with underline. Our method yields +10.6% average accuracy compared to the state-of-the-art.&lt;/em&gt;&lt;/p&gt;
&lt;h1 id=&#34;code&#34;&gt;Code&lt;/h1&gt;
&lt;p&gt;Our code and model checkpoints are publicly available here &lt;a href=&#34;https://github.com/mever-team/rine&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://github.com/mever-team/rine&lt;/a&gt;&lt;/p&gt;
&lt;h1 id=&#34;references&#34;&gt;References&lt;/h1&gt;
&lt;p&gt;Ojha, U., Li, Y., &amp;amp; Lee, Y. J. (2023). Towards universal fake image detectors that generalize across generative models. In &lt;em&gt;Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition&lt;/em&gt; (pp. 24480-24489).&lt;/p&gt;
&lt;p&gt;Wang, S. Y., Wang, O., Zhang, R., Owens, A., &amp;amp; Efros, A. A. (2020). CNN-generated images are surprisingly easy to spot&amp;hellip; for now. In &lt;em&gt;Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition&lt;/em&gt; (pp. 8695-8704).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Filtering external evidence for realistic training and evaluation of Automated Fact-Checking</title>
      <link>https://mever.gr/post/filtering-external-evidence-for-realistic-automated-fact-checking/</link>
      <pubDate>Wed, 03 Jul 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/filtering-external-evidence-for-realistic-automated-fact-checking/</guid>
      <description>&lt;p&gt;In this post, we explain the basics behind our paper &lt;a href=&#34;https://dl.acm.org/doi/abs/10.1145/3643491.3660278&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Credible, Unreliable or Leaked?: Evidence verification for enhanced automated fact-checking&lt;/a&gt; by Zacharias Chrysidis, Stefanos-Iordanis Papadopoulos, Symeon Papadopoulos and Panagiotis C. Petrantonakis, which has been presented at ICMR’s 3rd ACM International Workshop on Multimedia AI against Disinformation.&lt;/p&gt;
&lt;h2 id=&#34;automated-fact-checking-afc&#34;&gt;Automated Fact-Checking (AFC)&lt;/h2&gt;
&lt;p&gt;The Information Age, especially after the explosion of online platforms and social media, has led to a surge in new forms of mis- and disinformation, making it increasingly difficult for people to trust what they see and read online. To combat this, many fact-checking organizations, including &lt;a href=&#34;https://www.snopes.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Snopes&lt;/a&gt;, &lt;a href=&#34;https://www.politifact.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PolitiFact&lt;/a&gt; as well as &lt;a href=&#34;https://www.reuters.com/fact-check/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Reuters&lt;/a&gt; and &lt;a href=&#34;https://factcheck.afp.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AFP&lt;/a&gt; fact-checks, have emerged, dedicated to verifying claims in news articles and social media posts. Nevertheless, the manual process of fact-checking is time-consuming and can&amp;rsquo;t always keep pace with the rapid spread of mis- and disinformation. This is where the field of Automated Fact-Checking (AFC) comes in. In recent years, researchers have been trying to leverage advances in deep learning, large language models, computer vision, and multimodal learning to develop tools to assist the work of professional fact-checkers. AFC systems aim to automate key parts of the fact-checking process, such as detecting check-worthy claims, retrieving relevant evidence from the web, and cross-examining them against the claim (&lt;a href=&#34;https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00454/109469/A-Survey-on-Automated-Fact-Checking&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Guo, et al., 2022&lt;/a&gt;). Since fact-checking rarely relies solely on examining internal contradictions in claims, AFC systems often require the retrieval of external information from the web, knowledge databases, or performing inverse image searches to support or refute a claim, as shown below.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/evidence-retrieval-example.png&#34; alt=&#34;evidence_retrieval_example&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;An example of a claim, evidence collection and retrieval. Claim and verdict source: &lt;a href=&#34;https://www.snopes.com/fact-check/electric-bus-fire-kenya/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Snopes&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;challenges-of-training-automated-fact-checking-afc-systems&#34;&gt;Challenges of training Automated Fact-Checking (AFC) systems&lt;/h2&gt;
&lt;p&gt;Nevertheless, in order to develop and train AFC systems that can be applied to real-world cases, the collected evidence should follow certain criteria, such as “[the evidence in datasets] must be (1) sufficient to refute the claim and (2) not leaked from existing fact-checking articles” as defined by &lt;a href=&#34;https://aclanthology.org/2022.emnlp-main.397/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Glockner et al., 2022&lt;/a&gt;. The problem of leaked evidence in AFC systems arises when information from previously fact-checked articles is used during training. This can lead the model to overly rely on high-quality information curated by professional fact-checkers, making it less effective at handling new and emerging misinformation that has not yet been fact-checked –which is supposedly the task at hand. Additionally, the external information retrieved by the model must be credible to avoid feeding it with unreliable and false data. For example, as shown in the banner of this article, the retrieved evidence “Did President Obama Ban Muslims from Entering the United States in 2011?” is sourced from Snopes, thus, using the full article would allow leaked information into the AFC system. Conversely, feeding “MORE HYPOCRISY: Obama Banned all Iraqi Refugees for 6 Months in 2011 – Liberals SAID NOTHING” into the AFC system would introduce unreliable and false information. These problems are under-researched but crucial for achieving realistic and effective fact-checking.&lt;/p&gt;
&lt;h2 id=&#34;credible-unreliable-or-leaked-credule-dataset-assisting-the-detection-of-leaked-and-unreliable-evidence&#34;&gt;“CREDible, Unreliable, or LEaked” (CREDULE) dataset assisting the detection of leaked and unreliable evidence&lt;/h2&gt;
&lt;p&gt;To address these challenges, we developed the “CREDible, Unreliable, or LEaked” (CREDULE) dataset by modifying, merging, and extending previous datasets such as &lt;a href=&#34;https://www.copenlu.com/publication/2019_emnlp_augenstein/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;MultiFC&lt;/a&gt;, &lt;a href=&#34;https://www.kaggle.com/datasets/rmisra/politifact-fact-check-dataset&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PolitiFact&lt;/a&gt;, &lt;a href=&#34;https://github.com/neemakot/Health-Fact-Checking&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PUBHEALTH&lt;/a&gt;, &lt;a href=&#34;https://github.com/MELALab/nela-gt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NELA-GT&lt;/a&gt;, &lt;a href=&#34;https://github.com/several27/FakeNewsCorpus&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Fake News Corpus&lt;/a&gt;, and &lt;a href=&#34;https://www.kaggle.com/datasets/mrisdal/fake-news&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Getting Real About Fake News&lt;/a&gt;. The CREDULE dataset consists of 91,632 samples from 2016 to 2022, equally distributed across three classes: “Credible,” “Unreliable,” and “Fact-checked” (Leaked). Our criteria ensure balanced distributions not only across these classes but also across years, title length, and topics. The dataset includes both short texts (titles) and long texts (full articles), along with meta-data such as date, domain, URL, topic, and credibility scores. The credibility and bias scores were collected from the &lt;a href=&#34;https://mediabiasfactcheck.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Media Bias / Fact Check&lt;/a&gt; organization.&lt;/p&gt;
&lt;p&gt;After creating the CREDULE dataset, we developed the EVidence VERification Network (EVVER-Net), a neural network designed to detect leaked (fact-checked) and unreliable evidence during the evidence retrieval process. EVVER-Net can be integrated into an AFC pipeline to ensure that only credible and non-leaked information is used during training. The model leverages the power of large pre-trained Transformer-based text encoders, such as DeBERTa, CLIP, and T5 for short texts, and Long T5 and Longformer for long texts. It also integrates information from credibility and bias scores. EVVER-Net demonstrated impressive performance, achieving up to 89.0% accuracy without credibility scores and 94.4% with credibility scores.&lt;/p&gt;
&lt;h2 id=&#34;examining-the-evidence-of-widely-used-afc-datasets&#34;&gt;Examining the evidence of widely used AFC datasets&lt;/h2&gt;
&lt;p&gt;Finally, we applied the EVVER-Net model to the evidence provided by several widely used AFC datasets, including &lt;a href=&#34;https://github.com/Tariq60/LIAR-PLUS&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;LIAR-Plus&lt;/a&gt;, &lt;a href=&#34;https://github.com/VT-NLP/Mocheg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;MOCHEG&lt;/a&gt;, &lt;a href=&#34;https://github.com/Shreyashm16/Factify&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FACTIFY&lt;/a&gt;, &lt;a href=&#34;https://github.com/g-luo/news_clippings&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NewsCLIPpings&lt;/a&gt;, and &lt;a href=&#34;https://github.com/stevejpapad/image-text-verification&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;VERITE&lt;/a&gt;. EVVER-Net identified up to 98.5% of items in LIAR-Plus, 95.0% in the “Refute” class of FACTIFY, and 83.6% in MOCHEG as leaked evidence. For instance, in the FACTIFY dataset, the claim &amp;ldquo;Microsoft bought Sony for 121 billion&amp;rdquo; was accompanied by the evidence: &amp;ldquo;A satire article claiming that Microsoft has bought Sony for 130 billion has gone viral […] the original article in a Spanish website was intended to be a prank. [&amp;hellip;]&amp;rdquo; which is sourced from a previously fact-checked article and directly provides the model with the answer curated by a professional fact-checker. This limits the model&amp;rsquo;s ability to detect and verify new claims that have not yet been fact-checked.&lt;/p&gt;
&lt;p&gt;On the other hand, the NewsCLIPpings dataset had the lowest rate of leaked evidence at 14.1%, followed by &lt;a href=&#34;https://mever.gr/post/overcoming-unimodal-bias-in-multimodal-misinformation-detection/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;our own VERITE&lt;/a&gt; at 35.3%. For instance, a claim from VERITE: “An image shows a large crowd gathered on the Rio de Janeiro seafront to celebrate a Mass delivered by Pope Francis in 2013” is accompanied by evidence “Pope Francis wraps up Brazil trip with Mass for 3 million”, which is sourced by CBS News and is correctly identified as credible by EVVER-Net. Another claim, “A photograph captured by NASA’s Mars Curiosity Rover on May 7, 2022, showed an artificial portal nearby.” is accompanied by evidence “Does This NASA Photo Show a &amp;lsquo;Portal&amp;rsquo; and &amp;lsquo;Wall&amp;rsquo; on Mars?” sourced by  Snopes. The title by itself does not provide any leaked information into the model, but if the full article were to be fed into the model during training, it would reinforce the problem of leaked evidence.&lt;/p&gt;
&lt;p&gt;While NewsCLIPpings and VERITE display significant improvements over other datasets in the field, there is still room for enhancement, as 20% of the evidence in these datasets was estimated to be non-credible. By utilizing EVVER-Net or similar architectures, the credibility of the evidence in NewsCLIPpings and VERITE could be further improved, helping to avoid the issues of unreliable and leaked evidence and overall, building and evaluating AFC pipelines in a more realistic manner.&lt;/p&gt;
&lt;h2 id=&#34;references&#34;&gt;References&lt;/h2&gt;
&lt;p&gt;Guo, Z., Schlichtkrull, M., &amp;amp; Vlachos, A. (2022). A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10, 178-206.&lt;/p&gt;
&lt;p&gt;Glockner, M., Hou, Y., &amp;amp; Gurevych, I. (2022, December). Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (pp. 5916-5936).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SIDBench - A Python Framework for Reliably Assessing Synthetic Image Detection Methods</title>
      <link>https://mever.gr/post/sidbench-assessing-synthetic-image-detection-methods/</link>
      <pubDate>Fri, 21 Jun 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/sidbench-assessing-synthetic-image-detection-methods/</guid>
      <description>&lt;h2 id=&#34;the-problem&#34;&gt;The problem&lt;/h2&gt;
&lt;p&gt;Generative AI is making it easier to create incredibly realistic synthetic images. Detecting these fake images is challenging, and several methods have been developed to tackle this issue. However, there&amp;rsquo;s often a big difference between how well these methods work in experiments and how they perform in real life.&lt;/p&gt;
&lt;p&gt;To help bridge this gap, our recent work &lt;a href=&#34;https://arxiv.org/abs/2404.18552&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;SIDBench: A Python Framework for Reliably Assessing Synthetic Image Detection&amp;rdquo;&lt;/a&gt; introduces a new benchmarking framework that combines various state-of-the-art detection models. These models were chosen for their different features and network designs to cover a wide range of techniques. The framework uses the latest, highly realistic datasets and studies how common online image transformations, like JPEG compression, affect detection accuracy. This tool is available &lt;a href=&#34;https://github.com/mever-team/sidbench&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;online&lt;/a&gt; and is designed to be easily updated with new datasets and detection models, making it a flexible and valuable resource for improving synthetic image detection.&lt;/p&gt;
&lt;h2 id=&#34;the-sidbench-framework&#34;&gt;The SIDBench Framework&lt;/h2&gt;
&lt;p&gt;To map the current landscape of Synthetic Image Detection (SID) methods, we reviewed recent literature and identified 11 state-of-the-art models since 2020. These models were evaluated based on two criteria: the type of input used (raw images or extracted features like frequency patterns and noise) and the architecture of the model. Popular architectures include ResNet50 pre-trained on ImageNet and ViT-L/14 pre-trained with &lt;a href=&#34;https://arxiv.org/abs/2103.00020&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CLIP&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;main-findings-from-sidbench&#34;&gt;Main Findings from SIDBench&lt;/h2&gt;
&lt;h3 id=&#34;performance-across-different-models&#34;&gt;Performance Across Different Models&lt;/h3&gt;
&lt;p&gt;Our benchmark study verified that almost all models trained on GAN-generated images generalize well across different GANs. However, their performance drops significantly when applied to Diffusion Models (DMs), such as Guided Diffusion, Latent Diffusion, and Glide. Notably, some models like RINE and PatchCraft perform surprisingly well on DMs.&lt;/p&gt;
&lt;h3 id=&#34;calibration-for-improved-accuracy&#34;&gt;Calibration for Improved Accuracy&lt;/h3&gt;
&lt;p&gt;We discovered that accuracy could be further enhanced by calibrating the threshold used for classifying images. This suggests there is room for further improvement of performance.&lt;/p&gt;
&lt;h3 id=&#34;real-world-performance-evaluation&#34;&gt;Real-World Performance Evaluation&lt;/h3&gt;
&lt;p&gt;To assess real-world applicability, we evaluated ProGAN-trained models using the &lt;a href=&#34;https://zenodo.org/records/10066460&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Synthbuster&lt;/a&gt; dataset. Results showed a significant drop in performance across all models, highlighting the challenge of maintaining accuracy outside experimental conditions, with high-resolution, high quality images. The following table presents the overall accuracy for all proGAN-trained detectors.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/synthbuster-acc-all-models.png&#34; alt=&#34;synthbuster_results&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Accuracy (ACC) scores of integrated models for the &lt;a href=&#34;https://zenodo.org/records/10066460&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Synthbuster&lt;/a&gt; dataset&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&#34;impact-of-training-datasets&#34;&gt;Impact of Training Datasets&lt;/h3&gt;
&lt;p&gt;To explore the impact of training datasets, we included models trained on sources other than ProGAN. For instance, &lt;a href=&#34;https://arxiv.org/abs/2402.19091&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RINE&lt;/a&gt;, trained on Latent Diffusion, showed improved detection of DM images while retaining good performance on GAN-based images. Additionally, models trained on DM images yielded better results on challenging subsets of Synthbuster, such as images generated with the challenging Midjourney v5 algorithm.&lt;/p&gt;
&lt;h3 id=&#34;handling-image-preprocessing-resizing-and-cropping&#34;&gt;Handling Image Preprocessing (Resizing and Cropping)&lt;/h3&gt;
&lt;p&gt;When using pre-trained models, images can be either cropped to a predefined size or resized. Each method has its trade-offs: cropping can lose important information outside the cropped area, while resizing may introduce distortion, affecting performance. For example, resizing alters frequency components, which is crucial for models like FreqDetect that rely on frequency artifacts. Our findings indicate that, on average, performance remains relatively consistent regardless of the preprocessing method. However, the standard deviation in performance suggests that resizing versus cropping impacts each model differently across the Synthbuster datasets.&lt;/p&gt;
&lt;h3 id=&#34;influence-of-image-transformations&#34;&gt;Influence of Image Transformations&lt;/h3&gt;
&lt;p&gt;Image transformations like Gaussian blurring and JPEG re-compression negatively affect the performance of SID models. Our tests revealed that Gaussian blurring and JPEG re-compression generally reduce accuracy (ACC), with varying impacts on true positive rates (TPR) and true negative rates (TNR). Gaussian blurring tends to increase TPR at the expense of TNR, causing models to detect more images as fakes, including some real ones. However, when combined with JPEG re-compression, this behavior changes, indicating a need for further investigation. JPEG compression often impacts TNR less than TPR, except in some models where it leads to a complete inability to detect fake images.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/transformation-impact.png&#34; alt=&#34;transformation_impact&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;ACC, TPR and TNR across detection models for different image transformations. Metrics have been averaged across all evaluation datasets.&lt;/em&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Mitigating Viewer Impact From Disturbing Imagery Using AI Filters A User-Study</title>
      <link>https://mever.gr/post/mitigating-viewer-impact-from-disturbing-imagery-using-ai-filters-a-user-study/</link>
      <pubDate>Wed, 19 Jun 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/mitigating-viewer-impact-from-disturbing-imagery-using-ai-filters-a-user-study/</guid>
      <description>&lt;p&gt;In today&amp;rsquo;s digital age, professionals such as journalists and human rights investigators often encounter distressing and graphic imagery as part of their work. The emotional toll of viewing such content can be profound, leading to secondary trauma and affecting mental well-being. Traditional methods like blurring have been used to shield viewers from explicit details, but they often compromise image interpretability.&lt;/p&gt;
&lt;h2 id=&#34;exploring-the-capabilities-of-ai-filters&#34;&gt;Exploring The Capabilities of AI filters&lt;/h2&gt;
&lt;p&gt;Our recent paper &lt;a href=&#34;https://www.tandfonline.com/doi/full/10.1080/10447318.2024.2313890&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;Mitigating Viewer Impact From Disturbing Imagery Using AI Filters A User-Study&amp;rdquo;&lt;/a&gt; explores the effectiveness of Artificial Intelligence (AI) filters on mitigating the emotional impact of disturbing imagery while preserving crucial image information.&lt;/p&gt;
&lt;p&gt;AI has revolutionized various fields, including image processing. Style transfer techniques, which alter image appearance based on artistic styles, offer a promising alternative to traditional blurring. Unlike blurring, which can obscure details crucial for professional analysis, AI filters can maintain image integrity while reducing negative emotional responses significantly. Our study, involving 107 participants primarily from journalism and human rights fields, assessed the performance of five filter styles: Blurring, Partial Blurring, Drawing, Colored Drawing, and Painting.&lt;/p&gt;
&lt;h2 id=&#34;study-findings-the-power-of-ai-filters&#34;&gt;Study Findings: The Power of AI Filters&lt;/h2&gt;
&lt;p&gt;Participants reported a remarkable -30.38% reduction in negative emotional impact when viewing images filtered with the AI-based Drawing style. Moreover, this style retained an impressive 97.19% of image interpretability, far surpassing the 6.54% retained by traditional Blurring techniques. Furthermore, professionals highlighted the potential of AI filters as a preparatory step before viewing original images. This approach could mitigate immediate emotional distress while allowing for detailed inspection when necessary. Participants also suggested enhancements and identified workflow stages where AI filters could be seamlessly integrated.&lt;/p&gt;
&lt;p&gt;To delve deeper into our findings and explore the future of AI in content moderation, read our full study &lt;a href=&#34;https://www.tandfonline.com/doi/full/10.1080/10447318.2024.2313890&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Unveiling AI Bias in Facial Analysis: Meet FaceX</title>
      <link>https://mever.gr/post/facex-understanding-face-attribute-classifiers-through-summary-model-explanations/</link>
      <pubDate>Wed, 19 Jun 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/facex-understanding-face-attribute-classifiers-through-summary-model-explanations/</guid>
      <description>&lt;p&gt;In today&amp;rsquo;s world, AI systems are integral to many applications, from unlocking smartphones to recognizing emotions. While these advancements bring numerous benefits, they also raise ethical and societal concerns, particularly regarding AI bias. Facial analysis technologies, in particular, have been scrutinized for their potential biases, which can disproportionately affect certain groups of people. This is where our new tool, FaceX, comes into play.&lt;/p&gt;
&lt;h2 id=&#34;what-is-facex&#34;&gt;What is FaceX?&lt;/h2&gt;
&lt;p&gt;FaceX is a novel method designed to uncover biases in facial attribute classifiers. While existing instance-level XAI methods require labor-intensive manual inspections to draw general conclusions about model behavior, FaceX addresses this limitation by aggregating model activations at the region level, making it easier to visualize and understand biases. In particular, it summarizes model behavior across 19 predefined facial regions, such as hair, ears, and skin, providing a more comprehensive understanding of where and why a model focuses on specific facial areas.&lt;/p&gt;
&lt;h2 id=&#34;how-does-facex-work&#34;&gt;How Does FaceX Work?&lt;/h2&gt;
&lt;p&gt;FaceX computes a region-level aggregation of model instance-level attributions, summarizing the model’s output with respect to each region of interest. Then, spatial explanations, offered through a heatmap visualization over an abstract face prototype, provide in-depth understanding of the weight of each facial region (or accessory) on the model decision. Additionally, FaceX visualizes the high-impact image patches for each region, revealing not only where the model focuses but also helping the human analyst understand why certain features are influential. This dual approach of spatial explanation (understanding where the model focuses) and appearance-oriented insights (understanding the impact of specific image patches) sets FaceX apart as a powerful tool for identifying biases in facial analysis systems and acts as a comprehensive lens, allowing practitioners, researchers, and developers to scrutinize the entire spectrum of model behavior.&lt;/p&gt;
&lt;p&gt;Want to dive deeper? Check out the &lt;a href=&#34;https://dl.acm.org/doi/abs/10.1145/3652583.3658007&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper&lt;/a&gt; and &lt;a href=&#34;https://github.com/gsarridis/faceX&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;run&lt;/a&gt; FaceX to explain the behaviour of your models.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SDFD: Building a Versatile Synthetic Face Image Dataset with Diverse Attributes</title>
      <link>https://mever.gr/post/sdfd_building-a-versatile-synthetic-face-image-dataset-with-diverse-attributes/</link>
      <pubDate>Fri, 31 May 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/sdfd_building-a-versatile-synthetic-face-image-dataset-with-diverse-attributes/</guid>
      <description>&lt;p&gt;Artificial Intelligence (AI) systems today are incredibly advanced, leveraging vast amounts of data to perform complex tasks with high accuracy. These systems require huge datasets to train effectively. The quality and quantity of data significantly impact their performance, enabling them to recognize patterns, make predictions, and improve decision-making processes. As AI continues to evolve, the demand for large, diverse, and high-quality datasets increases especially, for image-based systems dealing with demographic attribute prediction that often face considerable bias and discrimination issues. Many current face image datasets primarily focus on representativeness in terms of demographic factors such as age, gender, and skin tone, overlooking other crucial facial attributes like hairstyle and accessories. This narrow focus limits the diversity of the data and consequently the robustness of AI systems trained on them.&lt;/p&gt;
&lt;p&gt;In our recent work &lt;a href=&#34;https://arxiv.org/abs/2404.17255&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;“SDFD: Building a Versatile Synthetic Face Image Dataset with Diverse Attributes”&lt;/a&gt; that we presented in the &lt;a href=&#34;https://responsiblefaceimageprocessing.github.io/fg2024/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;1st International Workshop on Responsible Face Image Processing&lt;/a&gt;, we tried to address the former limitations of existing datasets, by proposing a methodology for generating synthetic face image datasets that capture a broader spectrum of facial diversity. Specifically, our approach integrates a state-of-the-art text-to-image generation model with a systematic prompt formulation strategy, encompassing not only demographics and biometrics but also non-permanent traits like make-up, hairstyle, and accessories. These lead to a comprehensive dataset of high-quality realistic images, which can be used as an evaluation set in face analysis systems.&lt;/p&gt;
&lt;p&gt;The first step in our process was to compile a list of terms that will represent the attributes that one would like to be depicted in the target face images. These terms should be carefully picked and then filtered to eliminate specific words or phrases (see Table 1). After completing the basic attribute list, combinations of them are created to generate the text prompt. Prompts are used as input text to the generative model, describing the image that should be constructed. The more exact the description, the closer the resulting image will be to what was envisioned. Besides, negative terms are also important in the prompt process since they specify what should not be included in the generated image.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/sdfd_table.png&#34; alt=&#34;Table 1&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;em&gt;Table 1. Terms used in prompts. Options are presented in alphabetical order.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Figure 1 shows some example face image datasets created by our methodology along with their input prompts.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/sdfd_featured.png&#34; alt=&#34;Figure 1&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;em&gt;Fig. 1. Example generated images and their corresponding prompts without including universal prompt terms:a. Black, baby boy, short hair, 4K, Fujifilm XT3, b. Pacific Islander-Middle Eastern woman, having face tattoo, headband,having wrinkles, 8K, Nikon Z9, c. Pacific Islander, stressed, wearing headscarf, girl, black eyes, wearing lipstick, 8K, d. White, androgynous person, pink hair, blue eyes, Fujifilm XT3, e. White, girl, headband, Ultra HD, Nikon Z9, f. Southeast Asian-White, man, having piercing, dreadlocks, having wrinkles, Ultra HD, Canon Eos 5D.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;We opted for Stable Diffusion v2.1 [1] and generated 1000 face images, which constitute the first version of our Stabe Diffusion Face-image Dataset (SDFD).&lt;/p&gt;
&lt;p&gt;We compared SDFD with the existing FairFace [2] and LFW [3] datasets, which are established face images that comprise real images and are used in several fairness-aware research works. Our comparative study demonstrated that SDFD was an equally or even more challenging dataset compared to the existing ones, while also being considerably smaller in size.&lt;/p&gt;
&lt;p&gt;In the future, we intend to extend the number of images in our dataset to include less well-represented facial attributes such as scars, braces, and even disabilities - that we were unable to include in the current dataset version due to limitations in the text-to-image model. Besides, we intend to explore different prompting strategies and generative models. Finally, we would also like to extend SDFD with multiple images for the same identity to make it appropriate for other tasks such as face verification.&lt;/p&gt;
&lt;p&gt;SDFD is publicly available on &lt;a href=&#34;https://github.com/gebaltso/SDFD/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Github&lt;/a&gt; and the corresponding paper on &lt;a href=&#34;https://arxiv.org/abs/2404.17255&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;arXiv&lt;/a&gt;.&lt;/p&gt;
&lt;h1 id=&#34;references&#34;&gt;References&lt;/h1&gt;
&lt;p&gt;[1] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., &amp;amp; Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10684-10695).&lt;/p&gt;
&lt;p&gt;[2] Karkkainen, K., &amp;amp; Joo, J. (2021). Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp. 1548-1558).&lt;/p&gt;
&lt;p&gt;[3] Huang, G. B., Mattar, M., Berg, T., &amp;amp; Learned-Miller, E. (2008, October). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In Workshop on faces in &amp;lsquo;Real-Life&amp;rsquo; Images: detection, alignment, and recognition.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>FRCSyn-onGoing:Benchmarking and comprehensive evaluation of real and synthetic data to improve face recognition systems</title>
      <link>https://mever.gr/post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/</link>
      <pubDate>Fri, 03 May 2024 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/</guid>
      <description>&lt;p&gt;This post provides an overview of the paper &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S1566253524001003&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;FRCSyn-onGoing: Benchmarking and comprehensive evaluation of real and synthetic data to improve face recognition systems&amp;rdquo;&lt;/a&gt; , which has been published in the &lt;a href=&#34;https://www.sciencedirect.com/journal/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Information Fusion&lt;/a&gt;information-fusion journal. This has been a collaborative effort led by the organizers of the FRCSyn-onGoing challenge and several of the participating teams, such as ours.&lt;/p&gt;
&lt;p&gt;The utilization of synthetic data in face recognition systems offers several advantages over real-world datasets. Synthetic datasets provide a solution to privacy concerns associated with collecting real data that potentially include personal information without obtaining the consent by the affected individuals, a case that is quite common in image datasets sourced from the Internet. They also enable the generation of large datasets, which is especially important after the discontinuation of established databases due to privacy issues and regulations like the General Data Protection Regulation (GDPR) [27]. Synthetic data can be tailored to specific characteristics, such as demographic groups and age, without the need for additional human efforts, unlike real-world databases that may lack diversity in representation or suffer from a variety of sampling biases.&lt;/p&gt;
&lt;p&gt;In particular, the paper focuses on advancing face recognition technology by evaluating the effectiveness of using synthetic data alongside real data. This initiative involves a collaborative effort from researchers worldwide to address the limitations and challenges faced by current face recognition systems. By combining real and synthetic data, the paper aims to enhance the performance and accuracy of face recognition technology. The article summarizes the findings from the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) held at WACV 2024.&lt;/p&gt;
&lt;p&gt;One of the key aspects of the paper is the comprehensive evaluation of Face Recognition (FR) systems across different demographic groups and databases. Researchers have identified notable findings, such as lower performance on demographic groups representing the Asian population compared to others. However, training face recognition systems with synthetic data generated by frameworks like DCFace [1] and GANDiffFace [2] (see Figure 1) has led to promising results, particularly in addressing demographic diversity within test populations.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;Figure 1&#34; srcset=&#34;
               /post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/featured_hu_d4f3b7b69ff09ca8.webp 400w,
               /post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/featured_hu_8d7a593e7d1434ab.webp 760w,
               /post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/featured_hu_fd1937081eeb92c6.webp 1200w&#34;
               src=&#34;https://mever.gr/post/frcsyn_ongoing_benchmarking-and-comprehensive-evaluation-of-real-and-synthetic-data-to-improve-face-recognition-systems/featured_hu_d4f3b7b69ff09ca8.webp&#34;
               width=&#34;760&#34;
               height=&#34;185&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;em&gt;Figure 1: Examples of synthetic identities (one for each row) and their intra-class variations provided by two generative frameworks:(a) DCFace [1] and (b) GANDiffFa[2].&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Teams participating in the challenge, among which the MeVer team, proposed new approaches to reduce demographic bias and improve the overall performance of FR systems. By using ResNet backbones, ArcFace, and AdaFace loss functions, with several integrating multiple networks and advanced data augmentation techniques, researchers aimed to create less biased features and enhance the learning process for FR systems. Standardized benchmarks and metrics to evaluate FR performance in realistic scenarios, are also presented. By considering factors such as pose variations, aging, occlusions, and diverse demographic groups, researchers provide a more comprehensive assessment of the capabilities of these systems.&lt;/p&gt;
&lt;p&gt;Through the FRCSyn-onGoing challenge, researchers are paving the way for the future of FR technology by exploring the potential of synthetic data and domain generalization techniques. The collaboration between industry and academia in this initiative highlights the importance of addressing demographic bias and improving the fairness and accuracy of face recognition systems. By continuing to explore the benefits of synthetic data and innovative approaches, the paper aims to drive advancements in FR technology and contribute to rendering it more trustworthy and reliable.&lt;/p&gt;
&lt;h1 id=&#34;references&#34;&gt;References&lt;/h1&gt;
&lt;p&gt;[1] M. Kim, F. Liu, A. Jain, X. Liu, DCFace: Synthetic face generation with dual condition diffusion model, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.&lt;/p&gt;
&lt;p&gt;[2] P. Melzi, C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, D. Lawatsch, F. Domin, M. Schaubert, GANDiffFace: Controllable generation of synthetic datasets for face recognition with realistic variations, in: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2023.&lt;/p&gt;
&lt;p&gt;[3] P. Voigt, A. Von dem Bussche, The EU general data protection regulation (GDPR), in: A Practical Guide, first ed., Vol. 10, (3152676) Springer International Publishing, Cham, 2017, pp. 10–5555.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Identifying image provenance in the wild</title>
      <link>https://mever.gr/post/identifying-image-provenance-in-the-wild/</link>
      <pubDate>Wed, 09 Nov 2022 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/identifying-image-provenance-in-the-wild/</guid>
      <description>&lt;p&gt;In the era of smartphones and social media, where everyone is able to capture the moment and share it online in seconds, knowing the origin of an image could make the difference between early debunking of a fabricated story and the continued spread of a false message. In this blog post, we will take a look at the task of identifying the source of an image, a key problem in the field of image provenance analysis. Also, we will present our findings on how most of the big digital platforms modify the popular JPEG standard and how we use these traces for identifying the source of images that circulate online without any prior knowledge or assumption.&lt;/p&gt;
&lt;p&gt;The problem of image source identificationis in fact a long-standing one, challenging the people in the field of image forensics since the days digital photography started dominating the market. For example, one of the first considerable works in the area, attempting to identify the model of the camera used for shooting an image, dates back to &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/1418853&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;2004&lt;/a&gt; [1]. Following this work, various ideas have been proposed for tackling the same problem. The former mainly attempted to model the artifacts introduced by the image capture pipeline of each camera, ranging from the popular &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/1634362&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PRNU noise pattern&lt;/a&gt; [17] introduced by the imperfections of the camera sensor to the artifacts generated by the color filter array (CFA) [2]. However, in recent years the field has mostly moved to deep-learning based approaches, using state-of-the-art neural network architectures, like the &lt;a href=&#34;https://link.springer.com/article/10.1007/s00521-020-05220-y&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RemNet&lt;/a&gt; [3].&lt;/p&gt;
&lt;p&gt;However, the issue of source identification is not limited to finding the camera model used in shooting an image. An image may as well have been posted to various social media platforms or processed by an image editing software. In the field of &lt;a href=&#34;https://jis-eurasipjournals.springeropen.com/articles/10.1186/s13635-021-00117-2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;online media forensics&lt;/a&gt; [4], knowing the platform an image was uploaded to could be invaluable for tracking a disinformation campaign. Also, evidence that it was processed with some image editing software, like Adobe Photoshop, could turn suspicions that its contents have been tampered into certainty. For the former case, the image forensics community has presented a great amount of work that deals with the problem of source social media platform identification, like the work by &lt;a href=&#34;https://www.sciencedirect.com/science/article/abs/pii/S0167865521002257#bib0008&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Karunakar et al. (2021)&lt;/a&gt; [16]. On the other hand, we have seen a very limited amount of works that consider in some way the identification of the software used in tampering an image, like the more than decade old by &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S1742287608000285&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kornblum (2008)&lt;/a&gt; [12]. Instead, most of the available algorithms focus on localizing the tampered area of an image, like the ones provided by the &lt;a href=&#34;https://mever.gr/forensics/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;image verification assistant&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Furthermore, as &lt;a href=&#34;https://jis-eurasipjournals.springeropen.com/articles/10.1186/s13635-021-00117-2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Pasquini et al. (2021)&lt;/a&gt; [4] have noticed, many works in the source identification field are only able to discriminate between specific camera devices, without the ability to generalize for example to the rest of the devices of the same model. &lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-981-16-7621-5_7&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mandelli et al. (2022)&lt;/a&gt; [18] stated that this happens because most of these works utilize device-specific artifacts, like the popular &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/1634362&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PRNU pattern&lt;/a&gt; [17]. While these works could be useful in a juridical case, in the field of online media forensics it is rarely the case to have access to the specific camera device used in shooting a controversial photo.&lt;/p&gt;
&lt;h2 id=&#34;from-closed-set-to-open-set-identification&#34;&gt;From closed-set to open-set identification&lt;/h2&gt;
&lt;p&gt;Another aspect of the source identification task is the scope it is performed on. Most of the relevant works, regardless of the type of source they aim to identify, assume that all the possible sources are known beforehand and limit the task into selecting one of them. In reality, that means the possible options are limited to the ones contained in the datasets explored during the development of each work. This scenario is known as closed-set source identification and the latest algorithms seem to achieve accuracies easily exceeding 90%, as presented in the work of &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/1634362&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mandelli et al. (2022)&lt;/a&gt; [18]. However, in real-world media forensics, knowing the possible sources of an image beforehand is almost never the case.&lt;/p&gt;
&lt;p&gt;Due to the fact that an image found in the wild may have been captured by any smartphone or digital camera ever produced, modified by any of the available image editing software and shared through any of the existing social media platforms, some new ideas have emerged, attempting to deal with the task of open-set source identification, like the ones by &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/8732341&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Júnior et al. (2019)&lt;/a&gt; [5] and &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/9392307&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Lorch et al. (2021)&lt;/a&gt; [6]. The approaches taken in these cases ultimately attempt to answer two successive questions. The initial one is to decide whether the image under investigation comes from a set of known sources or not. Then, if the image has been found to originate from the group of known sources, the task is to attribute it to a specific source of the group, ultimately solving the closed-set problem. However, the evaluation of these algorithms has been primarily performed using the same available datasets, by hiding some of the included sources during the training phase, in an effort to model the group of unknown sources. So, considerable questions still remain on how well these methods would perform when having to deal with the much greater diversity encountered in the wild.&lt;/p&gt;
&lt;h2 id=&#34;identification-of-multiple-types-of-sources&#34;&gt;Identification of multiple types of sources&lt;/h2&gt;
&lt;p&gt;Multiple works in the field have relied primarily on the image signal for identifying the source. Also, the most sophisticated ones attempt to handle the unwanted artifacts introduced by subsequent stages in the lifetime of an image, in order to correctly identify the desired type of source. For example, &lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-981-15-7834-2_59&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sarkar et al. (2020)&lt;/a&gt; [8] proposed a technique more robust against the effects introduced by sharing an image to a social network platform, in order to correctly detect the source camera model. Others have attempted to identify the sequence of social media platforms an image may have been shared on, like &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/8683144&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Phan et al. (2019)&lt;/a&gt; [19] However, common ground on all of them is their intention to identify only a single type of source, e.g. only the camera or social network of origin. A key problem missed by these approaches is the ability to identify the last actor that operated on an image independently of its type (camera, social media platform, image editing software etc.), which could be an invaluable tool when tracking disinformation campaigns and examining images found on the web.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/exif_format.png&#34; alt=&#34;exif_format&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;The JPEG file format. Multiple of its components depend on the software stack used for saving an image. Image by Ozgur Ozcitak in codeproject.com available under MIT license.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;In a different approach, &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S174228761930026X&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mullan et al. (2019)&lt;/a&gt; [7] showed that the information contained in the &lt;a href=&#34;https://docs.fileformat.com/image/jpeg/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JPEG header&lt;/a&gt; is a powerful tool for estimating the source of an image, heavily characterizing the software stack used during compression. Also, the JPEG format currently dominates the industry, with the majority of digital cameras using it for saving the photos, while &lt;a href=&#34;https://w3techs.com/technologies/details/im-jpeg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;more than 3 out of 4 websites contain JPEG images&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;If some of the JPEG header data could be attributed with high certainty to some sources, this would allow deterministically attributing to their last source a great amount of the images that circulate on the web without a need for sophisticated signal processing.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/Examples-of-JPEG-quantization-tables-for-8-2-8-DCT-blocks-The-resulting-YCbCr-space-is.png&#34; alt=&#34;quantization_tables&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Examples of JPEG quantization tables. Image by Neelamani, Ramesh, et al. [10]&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;the-jpeg-quantization-tables&#34;&gt;The JPEG quantization tables&lt;/h2&gt;
&lt;p&gt;The &lt;a href=&#34;https://dl.acm.org/doi/abs/10.1145/103085.103089&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JPEG standard&lt;/a&gt; [9] defines a whole pipeline for compressing an image. The applied operations include among others the conversion of the image into the YCbCr colorspace, the splitting of it into 8×8 blocks, the application of the discrete cosine transform on each of them and the quantization operation. During quantization the 64 values inside each 8×8 block are divided by the values contained in an 8×8 quantization table and the decimal part of the results is stripped off. This process allows reducing the number of bits required for representing each frequency of the image by discarding some information, considering that the human eye is much more sensitive to some frequencies than others. Greater numbers in the quantization table mean that more information is discarded. Thus, the selection of the table greatly affects the quality of the compressed image. However, the standard itself does not define a specific table. Instead, it allows the users to use any tables suiting their needs and stores them into the header of the image. Also, different tables can be used for the quantization of each channel of the colorspace.&lt;/p&gt;
&lt;p&gt;A very common set of quantization tables is the one proposed by the &lt;a href=&#34;http://www.ijg.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Independent JPEG Group (IJG)&lt;/a&gt; for use in its libjpeg library, which many image software have adopted, so, as of today, it is considered as the standard one. The set is based on two tables, one for the luminance channel (Y) and one for the chrominance channels (Cr, Cb), which can be scaled, according to a provided formula, into 100 different quality levels.&lt;/p&gt;
&lt;p&gt;Based on the significance of the quantization table on image quality, it has been a long time since many companies in the photo industry, like digital camera and software brands, have started developing their own tables, tailored to the needs of their products. That for example means, a photo taken with a Canon camera would be distinguishable from another one taken with a Sony camera and both would be distinguishable before and after being edited with Adobe Photoshop in terms of their quantization tables. &lt;a href=&#34;https://digitalcommons.dartmouth.edu/cs_tr/291/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Farid (2006)&lt;/a&gt; [11] and &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S1742287608000285&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kornblum (2008)&lt;/a&gt; [12] were among the first to notice that the quantization table could be exploited for the purpose of source identification. However, they both focused only on a set of digital cameras of their era and on Adobe Photoshop.&lt;/p&gt;
&lt;h2 id=&#34;quantization-tables-in-the-imaging-industry&#34;&gt;Quantization tables in the imaging industry&lt;/h2&gt;
&lt;p&gt;In today’s image forensics, we still face the question on whether the use of quantization tables could be expanded for deterministically detecting images originating from popular social media platforms. Initially, given some older reports by &lt;a href=&#34;https://ieeexplore.ieee.org/abstract/document/8282271&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sun et al.&lt;/a&gt; in 2017 [13], stating that many popular social media platforms, including Facebook and Twitter, were utilizing only the IJG’s tables, we had limited hope. We could also verify that claim by examining the &lt;a href=&#34;https://jis-eurasipjournals.springeropen.com/articles/10.1186/s13635-017-0067-2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;VISION&lt;/a&gt; [14] dataset, which also dates back to 2017 and contains photos from multiple smartphones before and after being published to various social media platforms.&lt;/p&gt;
&lt;p&gt;However, moving on to the most recent publicly available dataset we were aware of, the &lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-3-030-68780-9_40&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FODB&lt;/a&gt; [15] which was released in 2020, we encountered a largely different situation. Images of 4 out of the 5 included social media platforms, namely Facebook, Instagram, Twitter and WhatsApp, were using custom quantization tables.&lt;/p&gt;
&lt;p&gt;Regarding smartphones, we also had signs of a similar trend. In VISION, we had noticed that only 2 out of the 11 manufacturers of the included models were utilizing their own quantization tables. On the other hand, in FODB, almost half of the included smartphone manufacturers were using non-standard tables.&lt;/p&gt;
&lt;p&gt;However, a big issue for our research was that even the most recent publicly available datasets mostly contained devices announced more than five years ago. We clearly could not rely on such data for framing the current situation of the market. So, we collected image samples for the 50 most popular smartphone models of 2021 and 2022 according to &lt;a href=&#34;https://mailchi.mp/counterpointresearch.com/smartphone-quarterly-global-q2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CounterPoint’s report&lt;/a&gt; and &lt;a href=&#34;https://www.flickr.com/cameras&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Flickr’s popularity index&lt;/a&gt;. Examining those data, we found out that brands previously using their own quantization tables continued in the same way, while brands that previously did not, like Xiaomi, Redmi and Google, started doing so.&lt;/p&gt;
&lt;p&gt;Additionally, from past works we already knew that digital camera and image editing software brands were among the first adopters of custom quantization tables. However, due to the age of these works, we had to revisit these areas to get a more accurate picture of today. So, we collected image samples for models of the top 8 camera brands in Flickr’s index. Also, we thoroughly examined all the export modes available to popular image editing software, including Adobe Photoshop, Corel PaintShop Pro and Serif Affinity Photo.&lt;/p&gt;
&lt;p&gt;Regarding digital cameras, we could not find a single model to not utilize manufacturer specific tables. Some brands, like Canon, use the same single table in multiple models, so it can clearly identify the manufacturer. Others, like Panasonic, seem to introduce some adaptivity to their table selection mechanism, which however generates a limited variety of tables, able to identify even some specific camera models.&lt;/p&gt;
&lt;p&gt;Finally, all of Adobe Photoshop, Corel PaintShop Pro and Serif Affinity Photo, utilize brand-specific tables. Especially in Photoshop, every single export mode utilizes Adobe’s own tables. So, if an image comes with them, we know that it has been processed in some way.&lt;/p&gt;
&lt;p&gt;So, we concluded that most big brands in the tech industry have introduced their own quantization tables in their products, which can be used for identifying the source of an image, in pretty acceptable granularities. Also, this trend seems to rise, since companies that previously were relying on the standard tables, are now starting to use their own.&lt;/p&gt;
&lt;h2 id=&#34;enhancements-to-the-image-verification-assistant&#34;&gt;Enhancements to the Image Verification Assistant&lt;/h2&gt;
&lt;p&gt;According to the findings we presented, we have updated the &lt;a href=&#34;https://mever.gr/forensics/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Image Verification Assistant&lt;/a&gt;, in order to be able to identify the source of images acquired in the wild, without any prior knowledge or assumption about their provenance. The assistant is able to identify the provenance of an uploaded photo in the following cases:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;social media platforms, namely Facebook, Instagram, WhatsApp and Twitter,&lt;/li&gt;
&lt;li&gt;image editing software and libraries, including Adobe Photoshop, Corel PaintShop Pro and Serif Affinity Photo,&lt;/li&gt;
&lt;li&gt;smartphones from the top brands in the market,&lt;/li&gt;
&lt;li&gt;digital cameras from all the established brands in the industry.
Based on our insights, the updated &lt;a href=&#34;https://mever.gr/forensics/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Image Verification Assistant&lt;/a&gt; is able to estimate the source of almost 80% of the submitted images with non-standard quantization tables. This should be considered a significant step forward from the previous version which could hardly exceed 20%. Below, we see the Assistant having detected the software that last processed a tampered image and the settings used for exporting it.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/source_identification_1-1024x421.jpg&#34; alt=&#34;verification_assistant&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;The Image Verification Assistant detected the software that last processed a tampered image and the settings used for exporting it. The text of the report has been zoomed in for better clarity. Screenshot of the Image Verification Assistant, edited by Dimitris Karageorgiou.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;references&#34;&gt;References&lt;/h2&gt;
&lt;p&gt;[1] Weyand, T., Kostrikov, I., &amp;amp; Philbin, J. (2016). Planet-photo geolocation with convolutional neural networks. In European Conference on Computer Vision (pp. 37-55).&lt;/p&gt;
&lt;p&gt;[1] Kharrazi, M., Sencar, H. T., &amp;amp; Memon, N. (2004). Blind source camera identification. In 2004 International Conference on Image Processing, 2004. ICIP’04. (Vol. 1, pp. 709-712). IEEE.&lt;/p&gt;
&lt;p&gt;[2] Cao, H., &amp;amp; Kot, A. C. (2010). Mobile camera identification using demosaicing features. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems (pp. 1683-1686). IEEE.&lt;/p&gt;
&lt;p&gt;[3] Rafi, A. M., Tonmoy, T. I., Kamal, U., Wu, Q. M., &amp;amp; Hasan, M. (2021). RemNet: remnant convolutional neural network for camera model identification. Neural Computing and Applications, 33(8), 3655-3670.&lt;/p&gt;
&lt;p&gt;[4] Pasquini, C., Amerini, I., &amp;amp; Boato, G. (2021). Media forensics on social media platforms: a survey. EURASIP Journal on Information Security, 2021(1), 1-19.&lt;/p&gt;
&lt;p&gt;[5] Júnior, P. R. M., Bondi, L., Bestagini, P., Tubaro, S., &amp;amp; Rocha, A. (2019). An in-depth study on open-set camera model identification. IEEE Access, 7, 180713-180726.&lt;/p&gt;
&lt;p&gt;[6] Lorch, B., Schirrmacher, F., Maier, A., &amp;amp; Riess, C. (2021). Reliable Camera Model Identification Using Sparse Gaussian Processes. IEEE Signal Processing Letters, 28, 912-916.&lt;/p&gt;
&lt;p&gt;[7] Mullan, P., Riess, C., &amp;amp; Freiling, F. (2019). Forensic source identification using JPEG image headers: The case of smartphones. Digital Investigation, 28, S68-S76.&lt;/p&gt;
&lt;p&gt;[8] Sarkar, B. N., Barman, S., &amp;amp; Naskar, R. (2021). Blind source camera identification of online social network images using adaptive thresholding technique. In Proceedings of International Conference on Frontiers in Computing and Systems (pp. 637-648). Springer, Singapore.&lt;/p&gt;
&lt;p&gt;[9] Wallace, G. K. (1991). The JPEG still picture compression standard. Communications of the ACM, 34(4), 30-44.&lt;/p&gt;
&lt;p&gt;[10] Neelamani, R., De Queiroz, R., Fan, Z., Dash, S., &amp;amp; Baraniuk, R. G. (2006). JPEG compression history estimation for color images. IEEE Transactions on Image Processing, 15(6), 1365-1378.&lt;/p&gt;
&lt;p&gt;[11] Farid, H. (2006). Digital image ballistics from JPEG quantization.&lt;/p&gt;
&lt;p&gt;[12] Kornblum, J. D. (2008). Using JPEG quantization tables to identify imagery processed by software. digital investigation, 5, S21-S25.&lt;/p&gt;
&lt;p&gt;[13] Sun, W., &amp;amp; Zhou, J. (2017). Image origin identification for online social networks (OSNs). In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (pp. 1512-1515). IEEE.&lt;/p&gt;
&lt;p&gt;[14] Shullani, D., Fontani, M., Iuliani, M., Shaya, O. A., &amp;amp; Piva, A. (2017). VISION: a video and image dataset for source identification. EURASIP Journal on Information Security, 2017(1), 1-16.&lt;/p&gt;
&lt;p&gt;[15] Hadwiger, B., &amp;amp; Riess, C. (2021). The Forchheim image database for camera identification in the wild. In International Conference on Pattern Recognition (pp. 500-515). Springer, Cham.&lt;/p&gt;
&lt;p&gt;[16] Karunakar, A. K., &amp;amp; Li, C. T. (2021). Identification of source social network of digital images using deep neural network. Pattern Recognition Letters, 150, 17-25.&lt;/p&gt;
&lt;p&gt;[17] Lukas, J., Fridrich, J., &amp;amp; Goljan, M. (2006). Digital camera identification from sensor pattern noise. IEEE Transactions on Information Forensics and Security, 1(2), 205-214.&lt;/p&gt;
&lt;p&gt;[18] Mandelli, S., Bonettini, N., &amp;amp; Bestagini, P. (2022). Source Camera Model Identification. In Multimedia Forensics (pp. 133-173). Springer, Singapore.&lt;/p&gt;
&lt;p&gt;[19] Phan, Q. T., Boato, G., Caldelli, R., &amp;amp; Amerini, I. (2019). Tracking multiple image sharing on social networks. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8266-8270). IEEE.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Online Disinformation in Europe</title>
      <link>https://mever.gr/post/online_disinformation_in_europe/</link>
      <pubDate>Tue, 24 May 2022 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/online_disinformation_in_europe/</guid>
      <description>&lt;p&gt;Last year, we had the opportunity to conduct a desk research study in an effort to map the landscape of online disinformation in Europe. Our research was guided by three research questions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;What are the measures and journalistic authorities in Europe that supervise and monitor the ethical application of journalism?&lt;/li&gt;
&lt;li&gt;Are there patterns of false information spreading as a means to serve the agenda and interests of certain political groups? Are there certain political groups that benefit as a result of given disinformation narratives?&lt;/li&gt;
&lt;li&gt;Are there specific and tangible policies that have been proposed by European and international organisations that could help tackle online disinformation?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To make the study of the first two questions more manageable, we focused on five European countries: France, Germany, Greece, Italy and Spain, and on three topics that are commonly associated with disinformation: COVID-19, immigration, and climate change. The resulting report is available here and we believe it could be valuable for informing a number of stakeholders, including professionals in the media industry and policy makers. In the following, we try to present the key insights that we managed to draw from this report.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;What are the measures and journalistic authorities in Europe that supervise and monitor the ethical application of journalism?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The primary organizations overseeing the application of ethical standards in journalism in Europe and worldwide are the &lt;a href=&#34;https://erga-online.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;European Regulators Group for Audiovisual Media Services&lt;/a&gt; (ERGA), the &lt;a href=&#34;https://europeanjournalists.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;European Federation of Journalists&lt;/a&gt; (EFJ) and the &lt;a href=&#34;https://www.poynter.org/ifcn/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;International Fact-Checking Network&lt;/a&gt; (IFCN). European countries follow their own national codes of ethical conduct, though there are certain commonalities across the conducts (cf. figure below) and truthful reporting is among these common principles.An additional important reference document in Europe is the &lt;a href=&#34;https://digital-strategy.ec.europa.eu/en/policies/code-practice-disinformation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Code of Practice on Disinformation&lt;/a&gt;, which includes relevant commitments on behalf of social media platforms and internet advertising companies. Finally, our study points out that complying with ethical standards is increasingly challenging for news organizations and journalists due to the 24/7 news lifecycle, the reliance on Internet advertising as a primary source of income and several other factors that affect media independence and plurality. Yet, failure to comply with ethical standards is found as a reason for the reduced trust of citizens in the media.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/codes_ethics.png&#34; alt=&#34;codes_ethics&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Summary of Codes of Ethics for EFJ and five studied European countries. Picture taken from &lt;a href=&#34;https://mever.gr/media/disinformation-study-v1.9.pdf&#34;&gt;Online Disinformation in Europe Report&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Are there patterns of false information spreading as a means to serve the agenda and interests of certain political groups? Are there certain political groups that benefit as a result of given disinformation narratives?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Among the three topics of focus for our study, disinformation was prevalent with respect to COVID-19 and immigration and much less pronounced with respect to climate change. At an EU and national level, we found evidence that COVID-19 related disinformation more often originates or is disseminated by far-right wing parties and politicians, while at an international level disinformation campaigns targeting European citizens appear to mainly originate from Russia and China and primarily target Germany and Italy. On the topic of immigration, we could identify several disinformation activities, featuring anti-immigrant narratives and sentiments, racist and xenophobic attitudes that were aligned with the agenda and ideology of far-right and rightwing parties, and sometimes pro-Kremlin media amplify the messages of far-right politicians (e.g. in Germany); however, there is no evidence of any kind of cooperation or coordination between them. On the topic of climate change, there seems to be a shift from climate change denialism to skepticism, but in general it appears that European countries are not fertile ground for climate change disinformation.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Are there specific and tangible policies that have been proposed by European and international organisations that could help tackle online disinformation?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;There are already a number of reports that recommend concrete policies and measures with a view to mitigating the effects of disinformation and limiting its root factors. Our analysis of existing recommendations highlights that the phenomenon of disinformation cannot be addressed with fragmented, one-dimensional or simply regulatory policies. It calls for a multi-dimensional, multi-faceted, multi-stakeholder policy framework that assigns fair responsibility to and requires decisive action from all relevant stakeholders. In particular, we present a six-dimensional policy framework (cf. figure below) that could be a useful reference for discussions among policy makers and other stakeholders. The recommended policies are organized in the following six dimensions: a) enhancement of the transparency of the digital media ecosystem; b) cultivation of media literacy and digital skills in different groups of citizens; c) empowerment of different groups of stakeholders, including platform users, citizens, and journalists; d) strengthening media independence and pluralism; e) promotion of ethical conduct in media, journalism and platforms; and f) support of independent research on monitoring the disinformation phenomenon and building services and tools for countering online disinformation.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/dis_framework.png&#34; alt=&#34;dis_framework&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Overview of six-dimensional policy framework to tackle disinformation. &lt;a href=&#34;https://mever.gr/media/disinformation-study-v1.9.pdf&#34;&gt;Picture taken from Online Disinformation in Europe Report&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The report was co-authored and researched by Ms. Georgia Pantalona, Dr. Filareti Tsalakanidou, Dr. Evangelia Kartsounidou, Dr. Symeon Papadopoulos, Dr. Nikos Sarris and Dr. Yiannis Kompatsiaris. The study was commissioned by the &lt;a href=&#34;https://left.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Left group in the European Parliament&lt;/a&gt; – GUE/NGL. To ensure the objectivity and transparency of the research, our research team conducted the study in a fully autonomous way without any intervention by the EP members. We made every possible effort to avoid expressing directly or indirectly any personal opinion or stance by the authors in this document and therefore we limited ourselves to analysing, synthesizing, summarizing and interpreting conclusions and evidence from primary sources. Last but not least, the researchers involved in the execution of the study have no direct or indirect association or affiliation with any political party or fraction. In addition, they have no further benefit or dependence on the content and conclusions of this study.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Deepfakes: An Emerging Internet Threat and their Detection</title>
      <link>https://mever.gr/post/deepfakes-an-emerging-internet-threat-and-their-detection/</link>
      <pubDate>Fri, 18 Dec 2020 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/deepfakes-an-emerging-internet-threat-and-their-detection/</guid>
      <description>&lt;p&gt;Towards the end of 2020, I had the opportunity to give a talk on Deepfakes in the 60th &lt;a href=&#34;https://www.ai4eu.eu/ai4eu-webcafes&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AI4EU WebCafe&lt;/a&gt;. With deepfakes being already with us for more than two years now, we have already witnessed several real-world cases of deepfakes with both positive and negative impact, while their use has recently been commoditized by popular apps such as ZAO and Reface. At the same time, it seems that there is an exponentially growing interest in the area from the academic community as indicated by the quickly growing number of papers related to the topic.&lt;/p&gt;
&lt;p&gt;Even though convincing deepfake generation is still hard and resource-intensive (both in terms of computing power and manual video editing skills), there has been immense progress in the area and numerous approaches and techniques that have led to remarkable improvements in the quality and realism of the resulting deepfakes, as well as on the requirements in terms of number of training samples and compute power. The following image shared by Ian Goodfellow, the father of Generative Adversarial Networks (GANs) vividly illustrates the progress achieved during the last years.&lt;/p&gt;
&lt;p&gt;TWEET&lt;/p&gt;
&lt;p&gt;What is more, there is now a variety of open-source tools and repositories, with &lt;a href=&#34;https://faceswap.dev/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FaceSwap&lt;/a&gt; and &lt;a href=&#34;https://github.com/iperov/DeepFaceLab&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFaceLab&lt;/a&gt; being the most popular ones, that have considerably lowered the barrier to entry even to non-experts (but still a bit tech-savvy).&lt;/p&gt;
&lt;p&gt;Following the progress of deepfake generation approaches, there has been commensurate progress in the inverse problem of deepfake detection. There have been numerous types of approaches tackling the problem. A first class of such approaches try to spot deepfake artifacts, much like a careful human inspector would, for instance, fuzzy and blurry areas around the lips, earlobes and hair, lack of symmetries, e.g. different colors between the left and right eye, fuzzy background, etc. There are also approaches that attempt to extract physiological signals from a video in question, e.g. eye blinking or human pulse, and detect whether there are inconsistencies or unrealistic patterns in their evolution. Other approaches adopt a more general supervised learning paradigm, trying to learn distinctive deepfake patterns by looking into sets of both deepfake and authentic images. Last but not least, as in the case of media forensics approaches, there are deepfake detection methods that try to uncover deepfake artifacts in the frequency domain.&lt;/p&gt;
&lt;p&gt;Even though the area is quite young, there are already a few very good surveys on the topic:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2001.06564&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Media Forensics and DeepFakes: an overview&lt;/a&gt; by Verdoliva&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2001.00179&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection&lt;/a&gt; by Tolosana et al.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://arxiv.org/abs/2004.11138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The Creation and Detection of Deepfakes: A Survey&lt;/a&gt; by Misky and Lee&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Another great landmark for deepfake detection research has been the launch and completion of the &lt;a href=&#34;https://www.kaggle.com/c/deepfake-detection-challenge&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepFake Detection Challenge&lt;/a&gt;. The challenge attracted more than 2100 participants and offered a very hard benchmarking setting, which revealed the main weakness of state-of-the-art deepfake detection methods, i.e. their limited ability to generalize to new types and forms of deepfake content. Yet, at the same time, the challenge pushed the performance boundary of solutions and raised awareness about new detection approaches to the community. Our MeVer team also participated in the challenge in the context of the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project and was quite successful, ranked among the top 5% of submitted solutions. We had already provided an overview of our solution in a &lt;a href=&#34;https://mever.gr/2020/06/16/mever-participation-in-deepfake-detection-challenge/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;previous blog post&lt;/a&gt;. Since then, we have also performed a number of improvements after releasing an alpha version of our service to our partners in the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project. You can get a glimpse on how our latest deepfake detection service looks in our latest &lt;a href=&#34;https://mever.gr/2020/12/17/mever-tools-for-image-and-video-verification-within-weverify/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blog post about verification tools&lt;/a&gt;, where we present all tools we developed so far in WeVerify.&lt;/p&gt;
&lt;p&gt;Closing, my thoughts on the current state of deepfakes are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Deepfake generation still requires a certain degree of expertise and the resulting deepfakes are in most cases still possible to detect by a trained human observer, but this is likely to change in the near future (2-3 years), making possible the creation of highly realistic deepfakes with very little manual intervention and computational resources.&lt;/li&gt;
&lt;li&gt;There are a variety of deepfake detection approaches, each working well on different kinds of deepfakes. A common issue among all of them is the limited generalization ability to cases that were not well represented in the training set. However, there are certain approaches that claim to be general (e.g. those based on analysis of physiological signals and analysis on the frequency domain), at least given the current state of deepfake generation technology.&lt;/li&gt;
&lt;li&gt;Currently, the negative impact of deepfakes has mainly affected individuals, e.g. revenge porn, cyberbullying, while there have also been cases where there was broader socio-political impact as a result of deepfakes or even the suspicion of deepfakes (e.g. Gabon president video).&lt;/li&gt;
&lt;li&gt;We shouldn’t be fooled by the fact that there has been limited use of deepfakes in the context of disinformation campaigns to date. This is likely not due to the limitations of the technology but probably due to the fact that there are already established disinformation techniques that still work effectively and do not require any technical expertise. In the near future, however, this could change, so the risk of deepfakes for disinformation should not be underestimated.
The video recording of the talk and the accompanying slides are available below.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;p&gt;SLIDES&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>MeVer tools for Image and Video Verification within WeVerify</title>
      <link>https://mever.gr/post/mever-tools-for-image-and-video-verification-within-weverify/</link>
      <pubDate>Thu, 17 Dec 2020 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/mever-tools-for-image-and-video-verification-within-weverify/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; is an EU granted project that aims to address the complex content verification challenges through a participatory verification approach, open-source algorithms, low-overhead human-in-the-loop machine learning and intuitive visualizations. The MeVer team leads the ‘Cross-modal Disinformation Detection and Content Verification’ work package and in collaboration with the University of Sheffield (USFD) have developed tools that will bring together features from text, images and videos to tackle the challenge of cross-modal disinformation detection and content verification.&lt;/p&gt;
&lt;p&gt;Having reached the end of the second year of the project and in view of the second year’s review we recorded demos of tools we built in the project to fulfil the needs of the WeVerify project.&lt;/p&gt;
&lt;p&gt;A brief overview and a demo on how to use and the features of each tool is provided below. For the tools that are password-protected, in case of interest, requests for trial should be made by email at &lt;a href=&#34;mailto:verifymedia@iti.gr&#34;&gt;verifymedia@iti.gr&lt;/a&gt; stating the purpose of use.&lt;/p&gt;
&lt;h2 id=&#34;deepfake-detector&#34;&gt;Deepfake detector&lt;/h2&gt;
&lt;p&gt;The deepfake detection algorithm processes media items (images or videos) and returns the probability that this media contains deepfake manipulated faces. In video cases, the video is segmented in shots and probabilities are extracted for every frame of the shots. The overall probability (video-level) is calculated by the shot with the higher deepfake probability.&lt;/p&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;h2 id=&#34;network-visualization&#34;&gt;Network Visualization&lt;/h2&gt;
&lt;p&gt;This is a web application that offers a network-based visualization of online social media conversations aiming to help users gain insights about how information spreads in the network and how communities are formed as a result of the groups frequently communicating with each other.&lt;/p&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;h2 id=&#34;image-verification-assistant&#34;&gt;Image Verification Assistant&lt;/h2&gt;
&lt;p&gt;The tool consists of a set of image forensics algorithms that try to find traces of forgeries in the query images. A metadata section is provided presenting all available metadata. A fusion method is already in place in order to combine the diverse outputs from the forensics algorithms of the tool.&lt;/p&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;h2 id=&#34;context-aggregation-and-analysis&#34;&gt;Context Aggregation and Analysis&lt;/h2&gt;
&lt;p&gt;The tool aims to facilitate the verification of user-generated images and videos from YouTube, Facebook and Twitter. It performs an analysis on the context surrounding the media item. It collects all publicly available metadata, analyzes them and creates a verification report.&lt;/p&gt;
&lt;p&gt;VIDEO&lt;/p&gt;
&lt;h2 id=&#34;near-duplicate-detection&#34;&gt;Near Duplicate detection&lt;/h2&gt;
&lt;p&gt;This tool provides reverse video search against a large collection of videos. The large collection of videos, called index, consists of the &lt;a href=&#34;https://github.com/MKLab-ITI/FIVR-200K&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FIVR-200K dataset&lt;/a&gt; and any other videos that are submitted by the users.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>MeVer meets DeepTrace</title>
      <link>https://mever.gr/post/mever-meets-deeptrace/</link>
      <pubDate>Tue, 10 Mar 2020 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/mever-meets-deeptrace/</guid>
      <description>&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/itideep.png&#34; alt=&#34;logos_itideep&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Polychronis from the MeVer team had the opportunity to visit &lt;a href=&#34;https://deeptracelabs.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DeepTrace Labs&lt;/a&gt;, a company based in Amsterdam, focusing on deepfake detection. MeVer and DeepTrace have recently started collaborating on the problem of DeepFake detection, which MeVer faces in the context of the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project.  As part of this collaboration, the two teams also joined forces to tackle the &lt;a href=&#34;https://www.kaggle.com/c/deepfake-detection-challenge&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kaggle’s DeepFake Detection Challenge&lt;/a&gt;. The goal of the challenge is to build effective solutions that can help detect DeepFakes and manipulated media. The final deadline for submissions is March 31, 2020.&lt;/p&gt;
&lt;p&gt;The first meeting took place at DeepTrace offices in Amsterdam on March 2, 2020. The meeting involved general discussion about the challenge, briefing about the current progress of each team, exchange of ideas for further improvements and plans for future work.&lt;/p&gt;
&lt;p&gt;Currently, there are more than 2000 teams participating in the competition and our detection models score among the top 30 teams. More updates coming soon.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Media and Journalism Students Play with MeVer Tools</title>
      <link>https://mever.gr/post/media-and-journalism-students-play-with-mever-tools/</link>
      <pubDate>Tue, 17 Dec 2019 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/media-and-journalism-students-play-with-mever-tools/</guid>
      <description>&lt;p&gt;The &lt;a href=&#34;https://caa.itigr/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Context Aggregation and Analysis&lt;/a&gt; tool was developed by the MeVer team within the InVID project and continues to evolve within the &lt;a href=&#34;https://weverify.eu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;WeVerify&lt;/a&gt; project. The tool aims to facilitate the verification of user-generated videos posted by three well-known platforms – YouTube, Facebook and Twitter. It collects information surrounding the video, analyses and filters it and creates a verification report that is then presented to the end user (journalist) who is responsible to inspect the verification cues and decide about the video veracity.&lt;/p&gt;
&lt;p&gt;To evaluate the value of the tool for journalistic verification tasks, a user study was designed and organized in collaboration with &lt;a href=&#34;https://media.jour.auth.gr/?facstaff=andreas-veglis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Professor Andreas Veglis&lt;/a&gt; and Dr. Dimitrios Giomelakis of the Media and Journalism department of the Aristotle University of Thessaloniki. On December 12, the user study was launched during the Media Informatics Lab attended by 28 students. The students, 9 male and 19 female, are at the 3rd or 4th year of their studies. MeVer member Olga Papadopoulou and also lead developer of the CAA tool, made a &lt;a href=&#34;https://www.slideshare.net/OlgaPapadopoulou/context-aggregation-and-analysis-a-tool-for-usergenerated-video-verification-206685791&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;presentation&lt;/a&gt; during the course, where she described the features and utility of the tool. Then, the students were asked to verify a set of fake and real videos and report the results using the CAA tool.&lt;/p&gt;
&lt;p&gt;PRESENTATION&lt;/p&gt;
&lt;p&gt;The user study has been split in three stages:&lt;/p&gt;
&lt;p&gt;-20 videos (fake and real) were provided to the students before the course and they were asked to verify them following best practices that they have already studied.
-During the course 10 videos (fake and real) were given to the students and they were asked to verify them using the CAA tool after the tool was presented to them.
-After the course the students were asked to verify 30 more videos (fake and real) using the CAA tool.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/during-the-study-3.jpg&#34; alt=&#34;during_study&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Students in action. Photo by Olga Papadopoulou.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/during-the-study.jpg&#34; alt=&#34;during_study&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Students in action. Photos by Dimitrios Giomelakis.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/during-the-study-2.jpg&#34; alt=&#34;during_study&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Students in action. Photos by Dimitrios Giomelakis.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The students were asked to report the results:
-A label of fake or real indicating the credibility of the video.
-The degree of certainty which quantifies how certain the student was about his/her label.
-The list of verification cues that contributed the verification (During the first stage, where the CAA tool was not used, the verification cues were not reported).
-A free text description of the analysis.
-The time that the student needed to verify the video.&lt;/p&gt;
&lt;p&gt;The study will run until January 13th, and then the collected data will be analyzed and reported.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://mever.gr/media/CAA-annotation2-511x1024.png&#34; alt=&#34;tool_screenshot&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

&lt;em&gt;Screenshot of the tool which was used for reporting the results.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The tool received good reviews and positive feedback from the students during its use and most of them showed interest and willingness to use it again. Olga Papadopoulou will visit the course on January 16, after the user study is completed, and she will have a discussion with the students in order to understand how useful the CAA tool has been to them and what needs to be improved to make it even more usable. The feedback will be taken into account in order to enhance the tool and release an improved version.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Video Verification in the Fake News Era</title>
      <link>https://mever.gr/post/video-verification-in-the-fake-news-era/</link>
      <pubDate>Wed, 25 Sep 2019 00:00:00 +0000</pubDate>
      <guid>https://mever.gr/post/video-verification-in-the-fake-news-era/</guid>
      <description>&lt;p&gt;MeVer has contributed to the editing and authoring of the brand new book &lt;a href=&#34;https://link.springer.com/book/10.1007/978-3-030-26752-0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;“Video Verification in the Fake News Era”&lt;/a&gt;  published by Springer last week.&lt;/p&gt;
&lt;p&gt;The book covers a range of recent technological advances and practical tools for discovering, verifying and visualizing social media video content, and managing related rights. In today’s increasingly disinformation-fraught media landscape, access to sophisticated editing and content management tools and the ease with which fake information spreads in electronic networks, require the entire news and media industries to carefully verify third-party content before publishing it.&lt;/p&gt;
&lt;p&gt;Our team authored or contributed to four chapters, each focusing on a technical approach that can be employed for tackling different parts of the verification
process:&lt;/p&gt;
&lt;p&gt;-&lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-3-030-26752-0_4&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Finding Near-Duplicate Videos in Large-scale Collections&lt;/a&gt; describes ways of identifying near-duplicate or highly similar videos compared to a video of interest (query) in very large video collections. This is of importance when one wants to quickly check whether a suspicious video already appeared in the past and was debunked.
-&lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-3-030-26752-0_5&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Finding Semantically Related Videos in Closed Collections&lt;/a&gt; presents  state-of-the-art approaches for two video retrieval tasks: a) accurately classifying videos into concepts, and b) identifying and recognizing logos shown in videos.
-&lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-3-030-26752-0_6&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Detecting Manipulations in Video&lt;/a&gt; offers a glimpse into the increasingly challenging problem of digital video manipulation and explores the potential of automatic detection methods based on deep learning.
-&lt;a href=&#34;https://link.springer.com/chapter/10.1007/978-3-030-26752-0_7&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Verification of Web Videos through Analysis of their Online Context&lt;/a&gt; presents an online tool that aims at assisting investigators with the verification of YouTube, Twitter and Facebook videos by producing a number of contextual bits of information and credibility cues.&lt;/p&gt;
&lt;p&gt;We consider the book to be of interest to computer scientists and researchers, news and media professionals, as well as policymakers and data-savvy media consumers.&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>
