Threshold-based Identification of Non-Gaussian Distortion in Optical Constellations Using Clustering Validity Metrics

Eduardo Avendano Fernandez; Henry Montaña Quintero; Lely Adriana Luenghas Contreras

doi:10.14482/inde.44.01.985.544

Authors

Eduardo Avendano Fernandez Universidad Pedagógica y Tecnológica de Colombia (UPTC) https://orcid.org/0000-0003-0910-8539
Henry Montaña Quintero Universidad Distrital Francisco José de Caldas https://orcid.org/0000-0003-0752-6315
Lely Adriana Luenghas Contreras Universidad Distrital Francisco José de Caldas https://orcid.org/0000-0002-3600-4666

DOI:

https://doi.org/10.14482/inde.44.01.985.544

Keywords:

Clustering, k-means, fuzzy c-means, Gustafson-Kessel, Non-Gaussian distortion, nonlinear phase noise

Abstract

In this work, we experimentally demonstrate a threshold-based demodulation strategy for 16-QAM and 4+12 PSK constellations affected by non-Gaussian distortions, using clustering validity indices as a decision metric. By applying fragmentation through clustering algorithms—k-means, fuzzy c-means (FCM), and Gustafson-Kessel FCM (GK-FCM)—we were able to identify ellipsoidal distortions on external data-symbol clusters and dynamically select appropriate demodulation strategies. The proposed clustering-based approach does not require IQ branch imbalance and phase-offset corrections by redefining decision regions based on cluster centroids. We introduce the use of clustering validity indexes (Partition Coefficient, Separation, Xie and Beni’s, and Dunn Index) to characterize symbol distortion levels in constellation diagrams and establish performance thresholds. The combination of DI and XB provides a criterion for defining the threshold of non-Gaussian distortion. In particular, XB ? 10.7 and DI ? 0.015 may serve as empirical indicators that the constellation in radio over fiber (RoF) optical systems has transitioned into a more structured regime where the cluster centroids are used for demodulation. Experimental results show that at high-noise levels (OSNR = 16 dB), the XB index reaches its minimum value, confirming the method’s sensitivity to noise-induced distortion. Improvements in the optical signal-to-noise ratio (OSNR) of up to 2.1 dB for 16-QAM and 0.7 dB for 4+12 PSK were observed at a BER threshold of 10?² after transmission over 78.8 km of fiber. The combination of DI and XB indices provides a robust criterion for defining the threshold of non-Gaussian distortion. These experimental findings suggest that clustering validity metrics can serve as effective thresholds for adaptive demodulation, enabling real-time identification of non-Gaussian distortions in RoF communication systems.

References

A. Dogra, R. K. Jha, and S. Jain, “A Survey on Beyond 5G Network With the Advent of 6G: Architecture and Emerging Technologies,” IEEE Access, vol. 9, pp. 67512–67547, 2021, doi: 10.1109/ACCESS.2020.3031234.

D. Marpaung, “High dynamic range analog photonic links design and simulation,” University of Twente, 2021.

J. Liu, X. Wu, C. Huang, H. K. Tsang, and C. Shu, “Compensation of Dispersion-Induced Power Fading in Analog Photonic Links by Gain-Transparent SBS,” IEEE Photonics Technol. Lett., vol. 30, no. 8, pp. 688–691, 2018, doi: 10.1109/LPT.2018.2812188.

V. A. Thomas, M. El-Hajjar, and L. Hanzo, “Performance Improvement and Cost Reduction Techniques for Radio Over Fiber Communications,” IEEE Commun. Surv. Tutorials, vol. 17, no. 2, pp. 627–670, 2015, doi: 10.1109/COMST.2015.2394911.

L. Manoliu, D. Wrana, B. Schoch, S. Haussmann, A. Tessmann, and I. Kallfass, “Frequency and Phase Investigation of the Local Oscillator Offset in a W-Band Satellite Communication Link,” in 2023 53rd European Microwave Conference (EuMC), 2023, pp. 360–363, doi: 10.23919/EuMC58039.2023.10290372.

L. Li, G. Zhang, X. Zheng, S. Li, H. Zhang, and B. Zhou, “Phase Noise Suppression for Single-Sideband Modulation Radio-Over-Fiber Systems Adopting Optical Spectrum Processing,” IEEE Photonics Technol. Lett., vol. 25, no. 11, pp. 1024–1026, 2013, doi: 10.1109/LPT.2013.2258901.

P. Li et al., “Multi-IF-Over-Fiber Based Mobile Fronthaul With Blind Linearization and Flexible Dispersion Induced Bandwidth Penalty Mitigation,” J. Light. Technol., vol. 37, no. 4, pp. 1424–1433, 2019, [Online]. Available: http://jlt.osa.org/abstract.cfm?URI=jlt-37-4-1424.

Y. Han, S. Yu, M. Li, J. Yang, and W. Gu, “An SVM-Based Detection for Coherent Optical APSK Systems With Nonlinear Phase Noise,” IEEE Photonics J., vol. 6, no. 5, pp. 1–10, 2014, doi: 10.1109/JPHOT.2014.2357424.

Y. Cui et al., “Overcoming Chromatic-Dispersion-Induced Power Fading in ROF Links Employing Parallel Modulators,” IEEE Photonics Technol. Lett., vol. 24, no. 14, pp. 1173–1175, 2012, doi: 10.1109/LPT.2012.2192422.

L. Pakala and B. Schmauss, “Two stage extended Kalman filtering for joint compensation of frequency offset, linear and nonlinear phase noise and amplitude noise in coherent QAM systems,” in 2017 19th International Conference on Transparent Optical Networks (ICTON), 2017, pp. 1–4, doi: 10.1109/ICTON.2017.8024972.

D. L. N. S. Inti, “Time-Varying Frequency Selective IQ Imbalance Estimation and Compensation,” Virginia Polytechnic Institute and State University, 2017.

M. Solarte-Sanchez, D. Marquez-Viloria, A. E. Castro-Ospina, E. Reyes-Vera, N. Guerrero-Gonzalez, and J. Botero-Valencia, “m-QAM Receiver Based on Data Stream Spectral Clustering for Optical Channels Dominated by Nonlinear Phase Noise,” Algorithms, vol. 17, no. 12. 2024, doi: 10.3390/a17120553.

D. Wang et al., “KNN-based detector for coherent optical systems in presence of nonlinear phase noise,” in 2016 21st OptoElectronics and Communications Conference (OECC) held jointly with 2016 International Conference on Photonics in Switching (PS), 2016, pp. 1–3.

J. J. Granada Torres, S. Varughese, S. E. Ralph, A. M. Cárdenas Soto, and N. G. González, “Clustering in Short Time Windows for Nonsymmetrical Demodulation in 16QAM Overlapped WDM Channels,” in Advanced Photonics 2017 (IPR, NOMA, Sensors, Networks, SPPCom, PS), 2017, p. SpM3F.2, doi: 10.1364/SPPCOM.2017.SpM3F.2.

A. F. Eduardo, G. T. Jhon James, C. S. Ana María, and G. G.-O. F. T. Neil, “Radio-over-Fiber Signal Demodulation in the Presence of Non-Gaussian Distortions based on Subregion Constellation Processing,” Opt. Fiber Technol., vol. 45IS–6, pp. 741–759, 2019, doi: https://doi.org/10.1016/eduardo.yofte.2019.08.004.

D. Lu et al., “100Gb/s PAM-4 VCSEL Driver and TIA for Short-Reach 400G-1.6T Optical Interconnects,” in 2021 IEEE Asia Pacific Conference on Circuit and Systems (APCCAS), 2021, pp. 253–256, doi: 10.1109/APCCAS51387.2021.9687808.

X. Guan, A. Omidi, M. Zeng, and L. A. Rusch, “Experimental Demonstration of a Constellation Shaped via Deep Learning and Robust to Residual-Phase-Noise,” in Conference on Lasers and Electro-Optics, 2022, p. SW4E.2, doi: 10.1364/CLEO_SI.2022.SW4E.2.

F. Ali, H. Afsar, A. Alshamrani, and A. Armghan, “Machine learning-based mitigation of thermal and nonlinear impairments in optical communication grids,” Opt. Laser Technol., vol. 182, p. 112090, 2025, doi: https://doi.org/10.1016/j.optlastec.2024.112090.

A. Kakkar et al., “Impact of local oscillator frequency noise on coherent optical systems with electronic dispersion compensation,” Opt. Express, vol. 23, no. 9, pp. 11221–11226, 2015, doi: 10.1364/OE.23.011221.

J. Zhang, M. Gao, W. Chen, and G. Shen, “Non-Data-Aidedk-Nearest Neighbors Technique for Optical Fiber Nonlinearity Mitigation,” J. Light. Technol., vol. 36, no. 17, pp. 3564–3572, 2018, doi: 10.1109/JLT.2018.2837689.

J. M. Cebrian, B. Imbernón, J. Soto, and J. M. Cecilia, “Evaluation of Clustering Algorithms on HPC Platforms,” Mathematics, vol. 9, no. 17. 2021, doi: 10.3390/math9172156.

D. E. Gustafson and W. C. Kessel, “Fuzzy clustering with a fuzzy covariance matrix,” in 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes, 1978, pp. 761–766, doi: 10.1109/CDC.1978.268028.

J. Lasserre, F. Ruiz, and T. Spina, “A double?suppressed possibilistic fuzzy Gustafson–Kessel clustering algorithm (DS?PFGK),” Knowledge-Based Syst., vol. 275, p. 110736, 2023, doi: 10.1016/j.knosys.2023.110736.

D.-W. Kim and K. H. Lee, “A new validity measure for fuzzy c-means clustering.” 2024, doi: 10.48550/arXiv.2407.06774.

P. Ghelfi, A. Bogoni, E. Avendaño Fernández, G. Serafino, A. M. Cardenas Soto, and N. Guerrero Gonzalez, “Machine Learning Techniques to Mitigate Nonlinear Phase Noise in Moderate Baud Rate Optical Communication Systems,” Y. (Cindy) Yi, Ed. Rijeka: IntechOpen, 2019.

B. Balasko, J. Abonyi, and B. Feil, “MATLAB files for ‘Manual for Fuzzy Clustering and Data Analysis Toolbox (For Use with Matlab).’” Jul. 07, 2014.

L. Vendramin, M. C. Naldi, and R. J. G. B. Campello, “Fuzzy Clustering Algorithms and Validity Indices for Distributed Data BT - Partitional Clustering Algorithms,” M. E. Celebi, Ed. Cham: Springer International Publishing, 2015, pp. 147–192.

E. A. Fernández, J. J. GranadaTorres, A. M. Cárdenas Soto, and N. G. González, “Geometric Constellation Shaping with Demodulation based-on Clustering to mitigate Phase-Noise in Radio-over-Fiber Systems,” in Latin America Optics and Photonics Conference, 2018, p. Tu5E.3, doi: 10.1364/LAOP.2018.Tu5E.3.

T. Zhao, A. Nehorai, and B. Porat, “K-means clustering-based data detection and symbol-timing recovery for burst-mode optical receiver,” IEEE Trans. Commun., vol. 54, no. 8, pp. 1492–1501, 2006, doi: 10.1109/TCOMM.2006.878840.

D. Wang et al., “Intelligent constellation diagram analyzer using convolutional neural network-based deep learning,” Opt. Express, vol. 25, no. 15, pp. 17150–17166, 2017, doi: 10.1364/OE.25.017150.

R. T. Jones et al., “Geometric Constellation Shaping for Fiber Optic Communication Systems via End-to-end Learning,” Oct. 2018, Accessed: Apr. 16, 2019. [Online]. Available: http://arxiv.org/abs/1810.00774.

M. A. Amirabadi, S. A. Nezamalhosseini, M. H. Kahaei, and L. R. Chen, “A Survey on Machine and Deep Learning for Optical Communications.” 2024, [Online]. Available: https://arxiv.org/abs/2412.17826.