ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
1.7k
Citations
200
Influential Citations
IEEE Transactions on Signal Processing
Venue
2021
Year
Recently, deep learned enabled end-to-end communication systems have been developed to merge all physical layer blocks in the traditional communication systems, which make joint transceiver optimization possible. Powered by deep learning, natural language processing has achieved great success in analyzing and understanding a large amount of language texts. Inspired by research results in both areas, we aim to provide a new view on communication systems from the semantic level. Particularly, we propose a deep learning based semantic communication system, named DeepSC, for text transmission. Based on the Transformer, the DeepSC aims at maximizing the system capacity and minimizing the semantic errors by recovering the meaning of sentences, rather than bit- or symbol-errors in traditional communications. Moreover, transfer learning is used to ensure the DeepSC applicable to different communication environments and to accelerate the model training process. To justify the performance of semantic communications accurately, we also initialize a new metric, named sentence similarity. Compared with the traditional communication system without considering semantic information exchange, the proposed DeepSC is more robust to channel variation and is able to achieve better performance, especially in the low signal-to-noise (SNR) regime, as demonstrated by the extensive simulation results.
Traditional communication systems are designed to minimize bit or symbol errors, but this does not guarantee that the intended meaning is preserved. As deep learning and natural language processing have advanced, there is a growing opportunity to rethink communication at the semantic level. This paper introduces DeepSC, a deep learning based semantic communication system for text transmission, which aims to recover the meaning of sentences rather than exact bits. This shift in perspective is significant because it aligns communication goals with human understanding, potentially enabling more efficient use of bandwidth and better performance in challenging channel conditions.
The work is timely given the increasing integration of AI into network infrastructure. By leveraging Transformer architectures and transfer learning, DeepSC demonstrates that semantic communication is feasible and can outperform traditional methods, especially when signal-to-noise ratio is low. This could have profound implications for future wireless systems, where reliability and efficiency are paramount.
The paper reports that DeepSC outperforms traditional communication systems that do not consider semantic information exchange. Specifically, it is more robust to channel variation and achieves better performance, especially in the low signal-to-noise (SNR) regime. The introduction of the sentence similarity metric allows for accurate evaluation of semantic preservation. While exact numerical results are not provided in the abstract, the extensive simulation results support the claims of improved performance.
This work bridges two fields—deep learning and communication systems—and opens a new research direction in semantic communication. By demonstrating that semantic-level transmission can be more robust and efficient, it challenges the traditional bit-centric approach. The use of Transformer and transfer learning highlights the potential of AI to revolutionize network design. This could lead to more efficient use of spectrum, better user experience, and new applications in machine-to-machine communication where meaning is more important than exact data reproduction. As 5G and beyond evolve, semantic communication may become a key technology for intelligent networks.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba