This paper focuses on an important type of black-box attacks, i.e., transfer-based adversarial attacks, where the adversary generates adversarial examples using a substitute (source) model and utilizes them to attack an unseen target model, without knowing its information. Existing methods tend to give unsatisfactory adversarial transferability when the source and target models are from different types of DNN architectures (e.g., ResNet-18 and Swin Transformer). In this paper, we observe that the above phenomenon is induced by the output inconsistency problem. To alleviate this problem while effectively utilizing the existing DNN models, we propose a common knowledge learning (CKL) framework to learn better network weights to generate adversarial examples with better transferability, under fixed network architectures. Specifically, to reduce the model-specific features and obtain better output distributions, we construct a multi-teacher framework, where the knowledge is distilled from different teacher architectures into one student network. By considering that the gradient of input is usually utilized to generate adversarial examples, we impose constraints on the gradients between the student and teacher models, to further alleviate the output inconsistency problem and enhance the adversarial transferability. Extensive experiments demonstrate that our proposed work can significantly improve the adversarial transferability.
The growing number of cases indicates that large language model (LLM) brings transformative advancements while raising privacy concerns. Despite promising recent surveys proposed in the literature, there is still a lack of comprehensive analysis dedicated to text privacy specifically for LLM. By comprehensively collecting LLM privacy research, we summarize five privacy issues and their corresponding solutions during both model training and invocation and extend our analysis to three research focuses in LLM application. Moreover, we propose five further research directions and provide prospects for LLM native security mechanisms. Notably, we find that most LLM privacy research is still in the technical exploration phase, with the hope that this work can assist in LLM privacy development.
Low-precision training has emerged as a practical approach, saving the cost of time, memory, and energy during deep neural networks (DNNs) training. Typically, the use of lower precision introduces quantization errors that need to be minimized to maintain model performance, often neglecting to consider the potential benefits of reducing training precision. This paper rethinks low-precision training, highlighting the potential benefits of lowering precision: (1) low precision can serve as a form of regularization in DNN training by constraining excessive variance in the model; (2) layer-wise low precision can be seen as an alternative dimension of sparsity, orthogonal to pruning, contributing to improved generalization in DNNs. Based on these analyses, we propose a simple yet powerful technique – DPC (Decreasing Precision with layer Capacity), which directly assigns different bit-widths to model layers, without the need for an exhaustive analysis of the training process or any delicate low-precision criteria. Thorough extensive experiments on five datasets and fourteen models across various applications consistently demonstrate the effectiveness of the proposed DPC technique in saving computational cost (−16.21%–−44.37%) while achieving comparable or even superior accuracy (up to +0.68%, +0.21% on average). Furthermore, we offer feature embedding visualizations and conduct further analysis with experiments to investigate the underlying mechanisms behind DPC’s effectiveness, enhancing our understanding of low-precision training. Our source code will be released upon paper acceptance.
Both accuracy and timeliness are key factors in detecting fake news on social media. However, most existing methods encounter an accuracy-timeliness dilemma: Content-only methods guarantee timeliness but perform moderately because of limited available information, while social context-based ones generally perform better but inevitably lead to latency because of social context accumulation needs. To break such a dilemma, a feasible but not well-studied solution is to leverage social contexts (e.g., comments) from historical news for training a detection model and apply it to newly emerging news without social contexts. This requires the model to (1) sufficiently learn helpful knowledge from social contexts, and (2) be well compatible with situations that social contexts are available or not. To achieve this goal, we propose to absorb and parameterize useful knowledge from comments in historical news and then inject it into a content-only detection model. Specifically, we design the Comments ASsisted FakENews Detection method (CAS-FEND), which transfers useful knowledge from a comment-aware teacher model to a content-only student model and detects newly emerging news with the student model. Experiments show that the CAS-FEND student model outperforms all content-only methods and even comment-aware ones with 1/4 comments as inputs, demonstrating its superiority for early detection.
Knowledge Graphs (KGs) are pivotal for effectively organizing and managing structured information across various applications. Financial KGs have been successfully employed in advancing applications such as audit, anti-fraud, and anti-money laundering. Despite their success, the construction of Chinese financial KGs has seen limited research due to the complex semantics. A significant challenge is the overlap triples problem, where entities feature in multiple relations within a sentence, hampering extraction accuracy – more than 39% of the triples in Chinese datasets exhibit the overlap triples. To address this, we propose the Entity-type-Enriched Cascaded Neural Network (E2CNN), leveraging special tokens for entity boundaries and types. E2CNN ensures consistency in entity types and excludes specific relations, mitigating overlap triple problems and enhancing relation extraction. Besides, we introduce the available Chinese financial dataset FINCORPUS.CN, annotated from annual reports of
Consistency identification in task-oriented dialogue (CI-ToD) can prevent inconsistent dialogue response generation, which has recently emerged as an important and growing research area. This paper takes the first step to explore a pre-training paradigm for CI-ToD. Nevertheless, pre-training for CI-ToD is non-trivial because it requires a large amount of multi-turn KB-grounded dialogues, which are extremely hard to collect. To alleviate the data scarcity problem for pre-training, we introduce a modularized pre-training framework (MPFToD), which is capable of utilizing large amounts of KB-free dialogues. Specifically, such modularization allows us to decouple CI-ToD into three sub-modules and propose three pre-training tasks including (i) query response matching pre-training; (ii) dialogue history consistent identification pre-training; and (iii) KB mask language modeling to enhance different abilities of CI-ToD model. As different sub-tasks are solved separately, MPFToD can learn from large amounts of KB-free dialogues for different modules, which are much easier to obtain. Results on the CI-ToD benchmark show that MPFToD pushes the state-of-the-art performance from 56.3% to 61.0%. Furthermore, we show its transferability with promising performance on other downstream tasks (i.e., dialog act recognition, sentiment classification and table fact checking).
Reconstructing from multi-view images is a longstanding problem in 3D vision, where neural radiance fields (NeRFs) have shown great potential and get realistic rendered images of novel views. Currently, most NeRF methods either require accurate camera poses or a large number of input images, or even both. Reconstructing NeRF from few-view images without poses is challenging and highly ill-posed. To address this problem, we propose CAD-NeRF, a method reconstructed from less than 10 images without any known poses. Specifically, we build a mini library of several CAD models from ShapeNet and render them from many random views. Given sparse-view input images, we run a model and pose retrieval from the library, to get a model with similar shapes, serving as the density supervision and pose initializations. Here we propose a multi-view pose retrieval method to avoid pose conflicts among views, which is a new and unseen problem in uncalibrated NeRF methods. Then, the geometry of the object is trained by the CAD guidance. The deformation of the density field and camera poses are optimized jointly. Then texture and density are trained and fine-tuned as well. All training phases are in self-supervised manners. Comprehensive evaluations of synthetic and real images show that CAD-NeRF successfully learns accurate densities with a large deformation from retrieved CAD models, showing the generalization abilities.
Function secret sharing (FSS) is a secret sharing technique for functions in a specific function class, mainly including distributed point function (DPF) and distributed comparison function (DCF). As an important basis for function secret sharing, DPF and DCF are the foundation for the extension of this technique to other more general and complex function classes. However, the function classes corresponding to the current DPF and DCF schemes are almost all unary function classes, and there is no efficient construction for multivariate function classes. The applications of FSS can be extended with the development of a multivariate scheme, e.g., a multi-keyword private information retrieval scheme can be constructed.
To solve this problem, this paper presents a binary DCF scheme based on the “two-layer binary tree” structure. In a binary tree structure, each node computes the seed of its child nodes based on its own seed. The key technique is to realize the transition transfer of seeds by using oblivious transfer, to connect two unary structures. Theoretical analysis and experimental results show that our binary scheme changes from single-round communication in the original definition to multi-round communication, and has great advantages in communication cost and computation efficiency. For the security parameter
In addition, we explore the extensions and applications of the above method. In the batch computation, this paper uses oblivious transfer (OT) extension to realize the one-time transmission of multiple pairs of seeds and optimize its communication efficiency. By extending the structure from “two-layer” to “multi-layer”, a secret sharing scheme of multivariate mixed basic function is proposed based on the serial thought. Furthermore, by employing the parallel thought, a general 2-layer FSS structure from OT for multivariate mixed basic functions is explored to enhance the efficiency, where the first layer is composed of d parallel binary trees with
Advanced Persistent Threats (APTs) pose significant challenges to detect due to their “low-and-slow” attack patterns and frequent use of zero-day vulnerabilities. Within this task, the extraction of long-term features is often crucial. In this work, we propose a novel end-to-end APT detection framework named Long-Term Feature Association Provenance Graph Detector (LT-ProveGD). Specifically, LT-ProveGD encodes contextual information of the dynamic provenance graph while preserving the topological information with space efficiency. To combat “low-and-slow” attacks, LT-ProveGD develops an autoencoder with an integrated multi-head attention mechanism to extract long-term dependencies within the encoded representations. Furthermore, to facilitate the detection of previously unknown attacks, we leverage Jenks’ natural breaks methodology, enabling detection without relying on specific attack information. By conducting extensive experiments on five widely used datasets with state-of-the-art attack detection methods, we demonstrate the superior effectiveness of LT-ProveGD.
