Deep learning-based joint source-channel coding (DJSCC) is expected to be a key technique for  {the} next-generation wireless networks. However, the existing DJSCC schemes still face the challenge of channel adaptability as they are typically trained under specific channel conditions. In this paper, we propose a generic framework for channel-adaptive DJSCC by utilizing hypernetworks. To tailor the hypernetwork-based framework for communication systems, we propose a memory-efficient hypernetwork parameterization and then develop a channel-adaptive DJSCC network, named Hyper-AJSCC. Compared with existing adaptive DJSCC based on the attention mechanism, Hyper-AJSCC introduces much fewer parameters and can be seamlessly combined with various existing DJSCC networks without any substantial modifications to their neural network architecture. Extensive experiments demonstrate the better adaptability to channel conditions and higher memory efficiency of Hyper-AJSCC compared with state-of-the-art baselines.

	@article{xie2024deep,
		title={Deep Learning-Based Adaptive Joint Source-Channel Coding using Hypernetworks},
		author={Xie, Songjie and He, Hengtao and Li, Hongru and Song, Shenghui and Zhang, Jun and Zhang, Ying-Jun Angela and Letaief, Khaled B},
		journal={arXiv preprint arXiv:2401.11155},
		year={2024}
	  }
	

	As machine learning (ML) becomes more prevalent in human-centric applications, there is a growing emphasis on algorithmic fairness and privacy protection. While previous research has explored these areas as separate objectives, there is a growing recognition of the complex relationship between privacy and fairness. However, previous works have primarily focused on examining the interplay between privacy and fairness through empirical investigations, with limited attention given to theoretical exploration. This study aims to bridge this gap by introducing a theoretical framework that enables a comprehensive examination of their interrelation. We shall develop and analyze an information bottleneck (IB) based information obfuscation method with local differential privacy (LDP) for fair representation learning. In contrast to many empirical studies on fairness in ML, we show that the incorporation of LDP randomizers during the encoding process can enhance the fairness of the learned representation. Our analysis will demonstrate that the disclosure of sensitive information is constrained by the privacy budget of the LDP randomizer, thereby enabling the optimization process within the IB framework to effectively suppress sensitive information while preserving the desired utility through obfuscation. Based on the proposed method, we further develop a variational representation encoding approach that simultaneously achieves fairness and LDP. Our variational encoding approach offers practical advantages. It is trained using a non-adversarial method and does not require the introduction of any variational prior. Extensive experiments will be presented to validate our theoretical results and demonstrate the ability of our proposed approach to achieve both LDP and fairness while preserving adequate utility.

	@article{xie2024privacy,
		title={Privacy for Fairness: Information Obfuscation for Fair Representation Learning with Local Differential Privacy},
		author={Xie, Songjie and Wu, Youlong and Li, Jiaxuan and Ding, Ming and Letaief, Khaled B},
		journal={arXiv preprint arXiv:2402.10473},
		year={2024}
	  }

Abstract arXiv Code BibTeX

	Deep learning-based joint source-channel coding (DJSCC) is expected to be a key technique for  {the} next-generation wireless networks. However, the existing DJSCC schemes still face the challenge of channel adaptability as they are typically trained under specific channel conditions. In this paper, we propose a generic framework for channel-adaptive DJSCC by utilizing hypernetworks. To tailor the hypernetwork-based framework for communication systems, we propose a memory-efficient hypernetwork parameterization and then develop a channel-adaptive DJSCC network, named Hyper-AJSCC. Compared with existing adaptive DJSCC based on the attention mechanism, Hyper-AJSCC introduces much fewer parameters and can be seamlessly combined with various existing DJSCC networks without any substantial modifications to their neural network architecture. Extensive experiments demonstrate the better adaptability to channel conditions and higher memory efficiency of Hyper-AJSCC compared with state-of-the-art baselines.

	@article{xie2024deep,
		title={Deep Learning-Based Adaptive Joint Source-Channel Coding using Hypernetworks},
		author={Xie, Songjie and He, Hengtao and Li, Hongru and Song, Shenghui and Zhang, Jun and Zhang, Ying-Jun Angela and Letaief, Khaled B},
		journal={arXiv preprint arXiv:2401.11155},
		year={2024}
	  }
	

Abstract arXiv BibTeX

	As machine learning (ML) becomes more prevalent in human-centric applications, there is a growing emphasis on algorithmic fairness and privacy protection. While previous research has explored these areas as separate objectives, there is a growing recognition of the complex relationship between privacy and fairness. However, previous works have primarily focused on examining the interplay between privacy and fairness through empirical investigations, with limited attention given to theoretical exploration. This study aims to bridge this gap by introducing a theoretical framework that enables a comprehensive examination of their interrelation. We shall develop and analyze an information bottleneck (IB) based information obfuscation method with local differential privacy (LDP) for fair representation learning. In contrast to many empirical studies on fairness in ML, we show that the incorporation of LDP randomizers during the encoding process can enhance the fairness of the learned representation. Our analysis will demonstrate that the disclosure of sensitive information is constrained by the privacy budget of the LDP randomizer, thereby enabling the optimization process within the IB framework to effectively suppress sensitive information while preserving the desired utility through obfuscation. Based on the proposed method, we further develop a variational representation encoding approach that simultaneously achieves fairness and LDP. Our variational encoding approach offers practical advantages. It is trained using a non-adversarial method and does not require the introduction of any variational prior. Extensive experiments will be presented to validate our theoretical results and demonstrate the ability of our proposed approach to achieve both LDP and fairness while preserving adequate utility.

	@article{xie2024privacy,
		title={Privacy for Fairness: Information Obfuscation for Fair Representation Learning with Local Differential Privacy},
		author={Xie, Songjie and Wu, Youlong and Li, Jiaxuan and Ding, Ming and Letaief, Khaled B},
		journal={arXiv preprint arXiv:2402.10473},
		year={2024}
	  }

Abstract arXiv Code Slides BibTeX

	Task-oriented communications, mostly using learning-based joint source-channel coding (JSCC), aim to design a communication-efficient edge inference system by transmitting task-relevant information to the receiver. However, only transmitting task-relevant information without introducing any redundancy may cause robustness issues in learning due to the channel variations, and the JSCC which directly maps the source data into continuous channel input symbols poses compatibility issues on existing digital communication systems. In this paper, we address these two issues by first investigating the inherent tradeoff between the informativeness of the encoded representations and the robustness to information distortion in the received representations, and then propose a task-oriented communication scheme with digital modulation, named discrete task-oriented JSCC (DT-JSCC), where the transmitter encodes the features into a discrete representation and transmits it to the receiver with the digital modulation scheme. In the DT-JSCC scheme, we develop a robust encoding framework, named robust information bottleneck (RIB), to improve the communication robustness to the channel variations, and derive a tractable variational upper bound of the RIB objective function using the variational approximation to overcome the computational intractability of mutual information. The experimental results demonstrate that the proposed DT-JSCC achieves better inference performance than the baseline methods with low communication latency, and exhibits robustness to channel variations due to the applied RIB framework. 

	@article{xie2023robust,
		title={Robust information bottleneck for task-oriented communication with digital modulation},
		author={Xie, Songjie and Ma, Shuai and Ding, Ming and Shi, Yuanming and Tang, Mingjian and Wu, Youlong},
		journal={IEEE Journal on Selected Areas in Communications},
		volume={41},
		number={8},
		pages={2577--2591},
		year={2023},
		publisher={IEEE}
	  }

Abstract PDF Code Slides BibTeX

	Recent work has explored federated clustering and developed an efficient $k$-means based method. However, it is well known that $k$-means clustering underperforms in high-dimensional space due to the so-called “curse of dimensionality". In addition, high-dimensional data (e.g., generated from healthcare, medical, and biological sectors) are pervasive in the big data era, which poses critical challenges to federated clustering in terms of, but not limited to, clustering effectiveness and communication efficiency. To fill this significant gap in federated clustering, we propose a one-shot federated subspace clustering scheme Fed-SC that can achieve remarkable clustering effectiveness on high-dimensional data while keeping communication cost low using only one round of communication for each local device. We further establish theoretical guarantees on the clustering effectiveness of one-shot Fed-SC and exploit the benefits of statistical heterogeneity across distributed data. Extensive experiments on synthetic and real-world datasets demonstrate significant effectiveness gains of Fed-SC compared with both subspace clustering and one-shot federated clustering methods.

@inproceedings{xie2023fed,
		title={Fed-SC: One-Shot Federated Subspace Clustering over High-Dimensional Data},
		author={Xie, Songjie and Wu, Youlong and Liao, Kewen and Chen, Lu and Liu, Chengfei and Shen, Haifeng and Tang, MingJian and Sun, Lu},
		booktitle={2023 IEEE 39th International Conference on Data Engineering (ICDE)},
		pages={2905--2918},
		year={2023},
		organization={IEEE}
	}

Abstract PDF BibTeX

	This paper focuses on K-receiver discrete-time memoryless broadcast channels (DM-BCs) with private messages, where the transmitter wishes to convey K private messages to K receivers. A general inner bound on the capacity region is proposed based on an exhaustive message splitting and a K-level modified Marton’s coding. The key idea is to split every message into ∑K ( K ) j=1 j−1 submessages each corresponding to a set of users who are assigned to recover them, and then send these submessages via codewords chosen from a K-level structure codebooks. To guarantee the joint typicality among all transmitted codewords, a sufficient condition on the subcodebooks' sizes is derived through a newly establishing hierarchical covering lemma, which extends the 2-level multivariate covering lemma to the K-level case with more intricate dependences. As the number of auxiliary random variables and rate conditions both increase exponentially with K, the standard Fourier–Motzkin elimination procedure becomes infeasible when K is large. To tackle this problem,	we obtain a closed form of achievable rate region with a special observation of disjoint unions of sets that constitute the power set of {1, . . . , K}. The proposed achievable rate region allows arbitrary input probability mass functions and improves over previously known achievable (closed form) rate regions for K-receiver (K ≥ 3) BCs. 

	@article{tang2021achievable,
		title={On the achievable rate region of the K-receiver broadcast channels via exhaustive message splitting},
		author={Tang, Rui and Xie, Songjie and Wu, Youlong},
		journal={Entropy},
		volume={23},
		number={11},
		pages={1408},
		year={2021},
		publisher={Multidisciplinary Digital Publishing Institute}
	  }