Acta Univ. Agric. Silvic. Mendelianae Brun. 2018, 66(2), 453-463 | DOI: 10.11118/actaun201866020453

Comparison of Methods for Smoothing Environmental Data with an Application to Particulate Matter PM10

Martina Čampulová
Department of Statistics and Operation Analysis, Faculty of Business and Economics, Mendel University in Brno, Zemědělská 1, 613 00, Brno, Czech Republic

Data smoothing is often required within the environmental data analysis. A number of methods and algorithms that can be applied for data smoothing have been proposed. This paper gives an overview and compares the performance of different smoothing procedures that estimate the trend in the data, based on the surrounding noisy observations that can be applied on environmental data.
The considered methods include kernel regression with both global and local bandwidth, moving average, exponential smoothing, robust repeated median regression, trend filtering and approach based on discrete Fourier and discrete wavelet transform. The methods are applied to real data obtained by measurement of PM10 concentrations and compared in a simulation study.

Keywords: data smoothing, trend filtering, environmental data, particulate matter PM10

Published: May 2, 2018  Show citation

ACS AIP APA ASA Harvard Chicago IEEE ISO690 MLA NLM Turabian Vancouver
Čampulová, M. (2018). Comparison of Methods for Smoothing Environmental Data with an Application to Particulate Matter PM10. Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, 66(2), 453-463. doi: 10.11118/actaun201866020453
Share...
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. ALSBERG, B., WOODWARDS, A. and KELL, D. 1997. An introduction to wavelet transforms for chemometricians: A time-frequency approach. Chemometrics and Intelligent Laboratory Systems, 37(2): 215-239. DOI: 10.1016/S0169-7439(97)00029-4 Go to original source...
    2. BARSANTI, R. J. and GILMORE, J. 2011. Comparing noise removal in the wavelet and Fourier domains. In: IEEE 43rd Southern Symposium on System Theory. 14 - 16 March, Auburn, AL, USA, pp. 163-167. Go to original source...
    3. BLEAKLEY, K. and VERT, J. P. 2011. The group fused Lasso for multiple change-point detection. [Online]. Available at: https://hal.archives-ouvertes.fr/hal-00602121 [Accessed: 2017, November 13].
    4. BOROWSKI, M. and FRIED, R. 2011. Robust repeated median regression in moving windows with data-adaptive width selection. Discussion paper 28/2011, SFB 823. Dortmund: Technische Universität.
    5. BOWERMAN, B. L., O'CONNELL, R. T. and KOEHLER, A. B. 2005. Forecasting, time series, and regression: an applied approach. 4th Edition. Belmont, CA: Thomson Brooks/Cole.
    6. CHEN, J. and GUPTA, A. 2012. Parametric statistical change point analysis: with applications to genetics, medicine, and finance. 2nd Edition. New York: Birkhäuser. Go to original source...
    7. COOK, E. and PETERS, K. 1981. The smoothing spline: a new approach to standardizing forest interior tree-ring width series for dendroclimatic studies. Tree-Ring Bulletin, 41: 45-55.
    8. ČAMPULOVÁ, M., VESELÍK, P. and MICHÁLEK, J. 2017. Control chart and six sigma based algorithms for identification of outliers in experimental data, with an application to particulate matter PM10. Atmospheric Pollution Research, 8(4): 700-708. DOI: 10.1016/j.apr.2017.01.004 Go to original source...
    9. DAUBECHIES, I. 1992. Ten Lectures on Wavelets. In: CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia: SIAM. Go to original source...
    10. DAVIES, P. L., FRIED, R. and GATHER, U. 2004. Robust signal extraction from on-line monitoring data, J. Statist. Plann. And Inference, 122: 65-78. DOI: 10.1016/j.jspi.2003.06.012 Go to original source...
    11. DONOHO, D. and JOHNSTONE, I. 1995. Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90(432): 1200-1244. DOI: 10.1080/01621459.1995.10476626 Go to original source...
    12. FANN, N., LAMSON, A., ANENBERG, S., WESSON, K., RISLEY, D. and HUBBELL, B. 2011. Estimating the national public health burden associated with exposure to ambient PM2 and ozone. Risk Analysis, 32(1): 81-95. DOI: 10.1111/j.1539-6924.2011.01630.x Go to original source...
    13. FRIED, R. 2004. Robust Filtering of Time Series with Trends. Journal of Nonparametric Statistics, 16(3-4): 313-328. DOI: 10.1080/10485250410001656444 Go to original source...
    14. FRIED, R. SCHETTLINGER, K. and BOROWSKI, M. 2014. robfilter: Robust Time Series Filters. R package version 4.1. Available at: https://CRAN.R-project.org/package=robfilter [Accessed: 2018, February 15].
    15. GASSER, T. and MÜLLER, H.-G. 1984. Estimating regression functions and their derivatives by the kernel method. Scandinavian Journal of Statistics, 11(3): 171-185.
    16. GASSER, T., MÜLLER, H.-G., and MAMMITZSCH, V. 1985. Kernels for nonparametric curve estimation. Journal of the Royal Statistical Society: Series B, 47(2): 238-252. Go to original source...
    17. GASSER, T., KNEIP, A. and KOEHLER, W. 1991. A flexible and fast method for automatic smoothing. Journal of the American Statistical Association, 86(415): 643-652. DOI: 10.1080/01621459.1991.10475090 Go to original source...
    18. GATHER, U. and FRIED, R. 2004. Methods and Algorithms for Robust Filtering. In: ANTOCH, J. (Ed.). COMPSTAT 2004: Proceedings in Computational Statistics. Berlin - Heidelberg: Physika-Verlag, pp. 159-170. Go to original source...
    19. HE, S., FANG, S., LIU, X., ZHANG, W., XIE, W., ZHANG, H., WEI, D., Fu, W. and PEI, D. 2016. Investigation of a genetic algorithm based cubic spline smoothing for baseline correction of raman spectra. Chemometrics and Intelligent Laboratory Systems, 152: 1-9. DOI: 10.1016/j.chemolab.2016.01.005 Go to original source...
    20. HENRY. R., NORRIS. G., VEDANTHAM, R. and TURNER, J. 2009. Source region identification using kernel smoothing. Environmental Science Technology, 43(11): 4090-4097. DOI: 10.1021/es8011723 Go to original source...
    21. HERRMANN, E. 1997. Local bandwidth choice in kernel regression estimation. Journal of Computational and Graphical Statistics, 6(1): 35-54. Go to original source...
    22. HERRMANN, E. 2014. Packaged for R and enhanced by Martin Maechler. lokern: Kernel Regression Smoothing with Local or Global Plug-in Bandwidth. R package version 1.1-6. [Online]. Available at: https://CRAN.R-project.org/package=lokern [Accessed: 2018, February 15].
    23. HOLE©OVSKÝ, H., ČAMPULOVÁ, M. and MICHÁLEK, J. 2017. Semiparametric outlier detection in nonstationary times series: Case study for atmospheric pollution in Brno, Czech Republic. Atmospheric Pollution Research, 9(1): 27-36. DOI: 10.1016/j.apr.2017.06.005 Go to original source...
    24. HOLT, C.C. 1957. Forecasting trends and seasonals by exponentially weighted moving averages, International Journal of Forecasting, 20(1): 5-10. DOI: 10.1016/j.ijforecast.2003.09.015 Go to original source...
    25. HRDLIČKOVÁ, Z., MICHÁLEK, J., KOLÁŘ and VESELÝ, V. 2008. Identifcation of factors affecting air pollution by dust aerosol PM10 in Brno city, Czech Republic. Atmospheric Environment, 42(37): 8661-8673. DOI: 10.1016/j.atmosenv.2008.08.017 Go to original source...
    26. HÜBNEROVÁ, Z. and MICHÁLEK, J. 2014. Analysis of daily average PM10 predictions by generalized linear models in Brno, Czech Republic. Atmospheric Pollution Research, 5(3): 471-476. DOI: 10.5094/APR.2014.055 Go to original source...
    27. HUTCHINSON, M. F. 1995. Interpolating mean rainfall using thin plate smoothing splines. International Journal of Geographic Information Systems, 9(4): 385-403. DOI: 10.1080/02693799508902045 Go to original source...
    28. HYNDMAN, R. J., KOEHLER, A. B., ORD, J. K. and SNYDER, R. D. 2008. Forecasting with Exponential Smoothing: The State Space Approach (Springer Series in Statistics). Berlin Heidelberg: Springer Verlag. Go to original source...
    29. KAFADAR, K. and MORRIS, M. 2002. Nonlinear smoothers in two dimensions for environmental data. Chemometrics and Intelligent Laboratory Systems, 60(1-2): 113-125. DOI: 10.1016/S0169-7439(01)00189-7 Go to original source...
    30. KANEKO, H. and FUNATSU, K. 2015. Smoothing-combined soft sensors for noise reduction and improvement of predictive ability. Industrial Engineering Chemistry Research, 54(50): 12630-12638. DOI: 10.1021/acs.iecr.5b03054 Go to original source...
    31. KIM, S. J., KOH, K., BOYD, S. and GORINEVSKY, D. 2009. L1 trend Filtering. SIAM Review, 51(2): 339-360. DOI: 10.1137/070690274 Go to original source...
    32. LEE, H., and KANG, K. 2015. Interpolation of missing precipitation data using kernel estimations for hydrologic modeling. Advances in Meteorology, 2015: 935868. Go to original source...
    33. LEVIN, D. 2015. Between Moving Least-Squares And Moving Least- ℓ1. Bit Numerical Mathematics, 55(3): 781-79. DOI: 10.1007/s10543-014-0522-0 Go to original source...
    34. MALLAT, S. 1998. A wavelet tour of signal processing. San Diego: Academic Press. Go to original source...
    35. MAMMEN, E. and GEER, S. 1997. Locally adaptive regression splines. Annals of Statistics, 25(1): 387-413. Go to original source...
    36. MEYER, Y. 1992. Wavelets and Operators. Cambridge: Cambridge University Press.
    37. NEUBAUER, J. and VESELY, V. 2011. Change point detection by sparse parameter estimation. Informatica, 22(1): 149-164. Go to original source...
    38. PAUL, S.K. 2011. Determination of Exponential Smoothing Constant to Minimize Mean Square Error and Mean Absolute Deviation. Global Journal of Research In Engineering: 11(3). Available at: https://engineeringresearch.org/index.php/GJRE/article/view/160
    39. PROAKIS, J. 1995. Digital Communications. New York: McGraw-Hill, Inc.
    40. RAJMIC, P., NOVOSADOVA, M. and DANKOVA, M. 2017. Piecewise-polynomial Signal Segmentation Using Convex Optimization. Kybernetika, 53(6): 1131-1149. Go to original source...
    41. RIDEL, B., GUENNEBAUD, G., REUTER, P., and GRANIER, X. 2015. Parabolic-Cylindrical Moving Least Squares Surfaces. Computers & Graphics, 51: 60-66. DOI: 10.1016/j.cag.2015.05.006 Go to original source...
    42. RUDIN, L. I., OSHER, S. and FATERNI, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1-4): 259-268. DOI: 10.1016/0167-2789(92)90242-F Go to original source...
    43. SANKARAN, S. EHSANI, R. and ETXEBERRIA, E. 2010. Mid-infrared spectroscopy for detection of huanglongbing (greening) in citrus leaves. Talanta, 83(2): 574-581. DOI: 10.1016/j.talanta.2010.10.008 Go to original source...
    44. SIEGEL, A. F. 1982. Robust signal extraction for on-line monitoring data. J. Statist. Plann. And Inference, 122: 65-78. Go to original source...
    45. DAVIES, P. L., FRIED, R. and GATHER, U. 2004. Robust signal extraction for on-line monitoring data. Journal of Statistical Planning and Inference, 122(1-2): 65-78. DOI: 10.1016/j.jspi.2003.06.012 Go to original source...
    46. STORLIE, C., BONDELL, H. and REICH, B. 2010. A locally adaptive penalty for estimation of functions with varying roughness. Journal of Computational and Graphical Statistics, 19(3): 569-589. DOI: 10.1198/jcgs.2010.09020 Go to original source...
    47. SVETUNOV, I. 2017. smooth: Forecasting Using Smoothing Functions. R package version 2.2.0. [Online]. Available at: https://CRAN.R-project.org/package=smooth [Accessed: 2018, February 15].
    48. TAYLOR, B.A. and TIBSHIRANI, R. 2014. genlasso: Path algorithm for generalized lasso problems. R package version 1.3. [Online]. Available at: https://CRAN.R-project.org/package=genlasso [Accessed: 2018, February 15].
    49. TIBSHIRANI, R., SAUNDERS, M., ROSSET, S., ZHU, J. and KNIGHT, K. 2005. Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society: Series B, 67(1): 91-108. DOI: 10.1111/j.1467-9868.2005.00490.x Go to original source...
    50. TIBSHIRANI, R. and TAYLOR, J. 2011. The solution path of the generalized lasso. Annals of Statistics, 39(3): 1335-1371. DOI: 10.1214/11-AOS878 Go to original source...
    51. TIBSHIRANI, R. 2014. Adaptive piecewise polynomial estimation via trend filtering. Annals of Statistics, 42(1): 285-323. DOI: 10.1214/13-AOS1189 Go to original source...
    52. TSAY, R. S. 2005. Analysis of financial time series. 2nd Edition. Hoboken: Wiley. Go to original source...
    53. TUKEY, J. W. 1977. Exploratory Data Analysis. Boston: Addison-Wesely.
    54. UNSER, M. 1999. Splines: A Perfect Fit For Signal And Image Processing. IEEE Signal Processing Magazine, 16(6): 22-38. DOI: 10.1109/79.799930 Go to original source...
    55. WALCZAK, B.and MASSART, D. 1997. Noise suppression and signal compression using the wavelet packet transform. Chemometrics and Intelligent Laboratory Systems, 36(2): 81-94. DOI: 10.1016/S0169-7439(96)00077-9 Go to original source...
    56. WAND, M.P. and JONES, M.C. 1995. Kernel Smoothing. London: Chapman and Hall. Go to original source...
    57. WANG, X., DU, P. and SHEN, J. 2013. Smoothing splines with varying smoothing parameter. Biometrika, 100(4): 955-970. DOI: 10.1093/biomet/ast031 Go to original source...
    58. WHITCHER, B. 2015. waveslim: Basic wavelet routines for one-, two- and three-dimensional signal processing. R package version 1.7.5. [Online]. Available at: https://CRAN.R-project.org/package=waveslim [Accessed: 2018, February 15].
    59. ZHANG, B., GENG J. and LAI, L. 2015. Multiple change-points estimation in linear regression models via sparse group lasso. IEEE Transactions on Signal Processing, 63(9): 2209-2224. DOI: 10.1109/TSP.2015.2411220 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY NC ND 4.0), which permits non-comercial use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content