Comprehensive methodology for cleaning and exploring quadcopter telemetry data: detection of missing and outlier values
DOI:
https://doi.org/10.36825/RITI.13.32.005Keywords:
Quadcopters, Telemetry, Data Cleaning, Exploratory Analysis, OutliersAbstract
This study presents an integrated methodology for cleaning and exploratory analysis of telemetry data from quadcopters, whose large volume and sensitivity to noise require rigorous processing to ensure reliability. The aim is to identify and correct missing and outlier values while characterizing relationships and distributions among key variables. Statistical techniques such as interpolation, adaptive winsorization, and visual verification were employed, complemented by exploratory data analysis using descriptive statistics, correlations (Pearson, Spearman, and partial), mutual information, and advanced visualizations (histograms, scatter plots, and pairplots). Results show complete removal of extreme values without significant loss of information, preserving the structural integrity of time series. The EDA revealed moderate-to-strong correlations among motor variables and nonlinear dependencies with IMU sensor signals, highlighting complex patterns relevant for subsequent modeling. The methodology provides a robust, reproducible framework applicable in similar contexts, establishing a solid foundation for predictive and control studies in UAVs. Future research will integrate explainable machine learning models to capture and interpret the detected interactions.
References
Zhao, J. (2023). Quadrotor’s modeling and control system design based on PID control. Journal of Physics: Conference Series, 2483, 1-13. https://doi.org/10.1088/1742-6596/2483/1/012034
Thanuja, K., Ravi Kumar, K. N., Ashwini, P., Chaithra, R., Sindhu, C. K., Varshitha, M. P. (2025). Development of unmanned aerial vehicle (UAV) for agricultural plant analysis. International Journal of Scientific Research in Engineering and Management, 9 (5), 1-3. https://doi.org/10.55041/ijsrem47026
Skazhennik, M. A., Chizhikov, V. N., Shevchenko, A., Migachev, A. N. (2021). Rice crops research according to remote sensing data (overview). E3S Web of Conferences, 285, 1-11. https://doi.org/10.1051/e3sconf/202128502038
Bazrafkan, A., Delavarpour, N., Oduor, P. G., Bandillo, N., & Flores, P. (2023). An overview of using unmanned aerial system–mounted sensors to measure plant above-ground biomass. Remote Sensing, 15 (14), 1-38. https://doi.org/10.3390/rs15143543
Rathod, P. D., Shinde, G. U. (2023). Autonomous aerial system (UAV) for sustainable agriculture: A review. International Journal of Environment and Climate Change, 13 (8), 1343-1355. https://doi.org/10.9734/ijecc/2023/v13i82080
Ameli, Z., Aremanda, Y., Friess, W. A., Landis, E. N. (2022). Impact of UAV hardware options on bridge inspection mission capabilities. Drones, 6 (3), 1-20. https://doi.org/10.3390/drones6030064
Bayomi, N., Fernández, J. (2023). Eyes in the sky: Drones applications in the built environment under climate change challenges. Drones, 7 (10), 1-42. https://doi.org/10.3390/drones7100637
Wu, R., Chao, W., Zhour, H. (2023). Research on hovering control system of four-rotor UAV in indoor environment. 3rd International Conference on Internet of Things and Smart City (IoTSC). Chongqing, China. https://doi.org/10.1117/12.2684049
Kumar, A., Yoon, S. (2020). Development of fast and soft landing system for quadcopter drone using fuzzy logic technology. International Journal of Advanced Trends in Computer Science and Engineering, 9 (1), 624-629. https://doi.org/10.30534/ijatcse/2020/87912020
Pentrakan, A., Chen, A. L. P. (2023). Data cleaning in medical procurement database: Performance comparison of data mining classification algorithms for tackling missing value. The Eurasia Proceedings of Science, Technology, Engineering and Mathematics, 23, 26-33. http://www.epstem.net/en/pub/issue/79793/1357602
Shi, X., Prins, C., Van Pottelbergh, G., Mamouris, P., Vaes, B., De Moor, B. (2021). An automated data cleaning method for electronic health records by incorporating clinical knowledge. BMC Medical Informatics and Decision Making, 21, 1-10. https://doi.org/10.1186/s12911-021-01630-7
Borrohou, S., Fissoune, R., Badir, H. (2023). Data cleaning survey and challenges: Improving outlier detection algorithm in machine learning. Journal of Smart Cities and Society, 2 (3), 125-140. https://doi.org/10.3233/SCS-230008
Guo, M., Wang., Y., Yang, Q., Li, R., Zhao, Y., Li, C., Zhu, M., Cui, Y., Jiang, X., Sheng, S., Li, Q., Gao, R. (2023). Normal workflow and key strategies for data cleaning toward real-world data: Viewpoint. Interactive Journal of Medical Research, 12, 1-11. https://doi.org/10.2196/44310
Sim, Y.-S., Hwang, J.-S., Mun, S.-D., Kim, T., Chang, S. J. (2022). Missing data imputation algorithm for transmission systems based on multivariate imputation with principal component analysis. IEEE Access, 10, 83195-83203. https://doi.org/10.1109/ACCESS.2022.3194545
Makarious, M. B., Leonard, H. L., Vitale, D., Iwaki, H., Saffo, D., Sargent, L., Dadu, A., Salemerón Castaño, E., Carter, J. F., Maleknia, M., Botia, J. A., Blauwendraat, C., Campbell, R. H., Hashemi, S. H., Singleton, A. B., Nalls, M. A., Faghri, F. (2021). GenoML: Automated machine learning for genomics. arXiv preprint. https://doi.org/10.48550/arXiv.2103.03221
Liu, Y., Jiang, X., Liu, P., Li, S. (2024). Data cleaning method based on multiple interpolation. Research Square (preprint). https://doi.org/10.21203/rs.3.rs-4866672/v1
Castiblanco Quintero, J. M., Garcia-Nieto, S., Simarro, R., Ignatyev, D. I. (2024). Improving racing drones flight analysis: A data-driven approach using motion capture systems. Drones, 8 (12), 1-27. https://doi.org/10.3390/drones8120742
Marcinkevičs, R., Vogt, J. E. (2021). Interpretable models for Granger causality using self-explaining neural networks. arXiv preprint. https://arxiv.org/pdf/2101.07600
Folkestad, C., Wei, S. X., Burdick, J. W. (2021). Quadrotor trajectory tracking with learned dynamics: Joint Koopman-based learning of system models and function dictionaries. arXiv preprint. https://arxiv.org/pdf/2110.10341
Silvagni, M., Tonoli, A., Zenerino, E., Chiaberge, M. (2022). UAV fault detection methods: State of the art. Drones, 6 (11), 1-39. https://doi.org/10.3390/drones6110330
Fourlas, G. K., Karras, G. C. (2021). A survey on fault diagnosis and fault-tolerant control methods for unmanned aerial vehicles. Machines, 9 (9), 1-34. https://doi.org/10.3390/machines9090197
Lalem, M. S., Ouadah, M., Touhami, O. (2024). Anomaly detection in quadcopter systems using AI and vibration signal processing. Research Square (preprint). https://doi.org/10.21203/rs.3.rs-5695145/v1
Kalinin, A. A., Palanimalai, S., Zhu, J., Wu, W., Devraj, N., Ye, C., Ponarul, N., Husain, S. S., Dinov, I. C. (2022). SOCRAT: A dynamic web toolbox for interactive data processing, analysis and visualization. Information, 13 (11), 1-24. https://doi.org/10.3390/info13110547
Abbas, N., Abbas, Z., Zafar, S., Ahmad, N., Liu, X., Khan, S. S., Foster, E. D., Larkin, S. (2024). Survey of advanced nonlinear control strategies for UAVs: Integration of sensors and hybrid techniques. Sensors, 24 (11), 1-51. https://doi.org/10.3390/s24113286
Jeong, S. H., Kang, D., Lee, I., Lee, Y., Kim, J. H., Hwang, Y.-Y. (2024). Gap filling of missing and outlier values of rotorcraft flight data using multilayer perceptron. Preprints.org. https://doi.org/10.20944/preprints202405.1581.v1
Nugroho, H., Utama, N. P., Surendro, K. (2021). Normalization and outlier removal in class center-based firefly algorithm for missing value imputation. Journal of Big Data, 8, 1-18. https://doi.org/10.1186/s40537-021-00518-7
Ahn, H., Sun, K., Kim, K.-H. (2022). Comparison of missing data imputation methods in time series forecasting. Computers, Materials & Continua, 70 (1), 767-779. https://doi.org/10.32604/cmc.2022.019369
Kowalska-Styczeń, A., Owczarek, T., Siwy, J., Sojda, A., Wolny, M. (2022). Analysis of business customers’ energy consumption data registered by trading companies in Poland. Energies, 15 (14), 1-23. https://doi.org/10.3390/en15145129
Huyghues-Beaufond, N., Tindemans, S. H., Falugi, P., Sun, M., Štrbac, G. (2020). Robust and automatic data cleansing method for short-term load forecasting of distribution feeders. Applied Energy, 261. https://doi.org/10.1016/j.apenergy.2019.114405
Asanka, D., Takahashi, M., Rajapakshe, C. (2024). Improving human mobility forecasts: A study on outlier correction with multi-agent techniques. Research Square (preprint). https://doi.org/10.21203/rs.3.rs-5365189/v1
The pandas development team. (2025). Pandas (software release). Zenodo. https://doi.org/10.5281/zenodo.15597513
Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9 (3), 90–95. https://doi.org/10.1109/MCSE.2007.55
Waskom, M. L. (2021). seaborn: Statistical data visualization. Journal of Open Source Software, 6 (60), 1-4. https://doi.org/10.21105/joss.03021
Mishra, D. P., Kumar, P., Rai, P., Kumar, P., Salkuti, S. R. (2024). Exploratory data analysis for electric vehicle driving range prediction: Insights and evaluation. International Journal of Applied Power Engineering, 13 (2), 474-482. https://doi.org/10.11591/ijape.v13.i2.pp474-482
Rabbi, M. F., Kovács, S. (2024). Quantifying global warming potential variations from greenhouse gas emission sources in forest ecosystems. Carbon Research, 3 (70), 1-17. https://doi.org/10.1007/s44246-024-00156-7
Thavarajasingam, S. G., El-Khatib, M., Vemulapalli, K., Sinzinkayo Iradukunda, H. A., Vishnu, S., Borchert, R., Russo. S., Eide, P. K (2023). Radiological predictors of shunt response in idiopathic normal pressure hydrocephalus: A systematic review and meta-analysis. Acta Neurochirurgica, 165, 369-419. https://doi.org/10.1007/s00701-022-05402-8
Bae, S. H., Noh, Y., Seo, P. J. (2022). REGENOMICS: A web-based application for plant REGENeration-associated transcriptomics analyses. Computational and Structural Biotechnology Journal, 20, 3234-3247. https://doi.org/10.1016/j.csbj.2022.06.033
Hassan Baabbad, H. K., Artun, E., Kulga, B. (2022). Understanding the controlling factors for CO₂ sequestration in depleted shale reservoirs using data analytics and machine learning. ACS Omega, 7 (24), 20845-20859. https://doi.org/10.1021/acsomega.2c01445
Bassek, M., Raabe, D., Memmert, D., Rein, R. (2023). Analysis of motion characteristics and metabolic power in elite male handball players. Journal of Sports Science and Medicine, 22, 310-316. https://doi.org/10.52082/jssm.2023.310
Newburger, E., Correll, M., Elmqvist, N. (2023). Fitting bell curves to data distributions using visualization. IEEE Transactions on Visualization and Computer Graphics, 29 (12), 5372-5383. https://doi.org/10.1109/TVCG.2022.3210763
Bouqentar, M. A., Terrada, O., Hamida, S., Saleh, S., Lamrani, D., Cherradi, B., Raihani, A. (2024). Early heart disease prediction using feature engineering and machine learning algorithms. Heliyon, 10 (19), 1-23. https://doi.org/10.1016/j.heliyon.2024.e38731
Paliwoda, D., Mikiciuk, G., Chudecka, J., Tomaszewicz, T., Miller, T., Mikiciuk, M., Kisiel, A., Sas-Paszt, L. (2023). Effects of inoculation with plant growth-promoting rhizobacteria on chemical composition of the substrate and nutrient content in strawberry plants growing in different water conditions. Agriculture, 14 (1), 1-31. https://doi.org/10.3390/agriculture14010046
Correll, M. (2023). Teru Teru Bōzu: Defensive raincloud plots. Computer Graphics Forum, 42 (3), 235-246. https://doi.org/10.1111/cgf.14826
Adnan, M., Altalhi, M., Alarood, A. A., Uddin, M. I. (2022). Modeling the spread of COVID-19 by leveraging machine and deep learning models. Intelligent Automation & Soft Computing, 31 (3), 1857-1872. https://doi.org/10.32604/iasc.2022.020606
Kumar, Y., Koul, A., Kaur, S., Hu, Y.-C. (2022). Machine learning and deep learning based time-series prediction and forecasting of ten nations’ COVID-19 pandemic. SN Computer Science, 4 (91), 1-27. https://doi.org/10.1007/s42979-022-01493-3
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Revista de Investigación en Tecnologías de la Información
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Esta revista proporciona un acceso abierto a su contenido, basado en el principio de que ofrecer al público un acceso libre a las investigaciones ayuda a un mayor intercambio global del conocimiento.
El texto publicado en la Revista de Investigación en Tecnologías de la Información (RITI) se distribuye bajo la licencia Creative Commons (CC BY-NC
), que permite a terceros utilizar lo publicado citando a los autores del trabajo y a RITI, pero sin hacer uso del material con propósitos comerciales.
