Weak sparseness in statistics
Although we introduced the terminology later, the weak sparseness model is a pair of hypotheses that are instrumental in [A sharp upper bound] to prove Theorem VII.1 for the detection of a random signal with unknown distribution in independent Gaussian noise.
[A sharp upper bound] D. Pastor, R. Gay, A. Groenenboom. A sharp upper bound for the probability of error of the likelihood ratio test for detecting signals in white Gaussian noise. IEEE Transactions on Information Theory, Institute of Electrical and Electronics Engineers, 2002, 48 (1), pp.228-238. ⟨10.1109/18.971751⟩. ⟨hal-02194866⟩
These two assumptions are that:
-
The signal has probability of occurrence less than or equal to one half
-
The signal has minimum amplitude above some known lower bound.
We used these two hypotheses, without yet encapsulating them under the term of "weak sparseness model", to estimate the standard deviation of white Gaussian noise in presence of signals. We began with [A limit theorem], from which we derived a first estimator.
[A limit theorem] D.Pastor (2008). A theoretical result for processing signals that have unknown distributions and priors in white Gaussian noise. Computational Statistics and Data Analysis, 52(6):3167-3186. https://doi.org/10.1016/j.csda.2007.10.011
With some hindsight, I'm tempted to rewrite [A limit theorem] because it's rather intricate. A consequence of this intricateness is that the noise estimator derived in this paper performs well but is quite complex. We have thus looked for simpler versions. We found two of them, [Noise variance estimation] and [The DATE] (d-Dimensional Amplitude Trimmed Estimator).
[Noise variance estimation] F.-X. Socheleau, D. Pastor, A. Aissa El Bey (2011). Robust statistics based noise variance estimation: application to wideband interception of non-cooperative communications. IEEE Transactions on Aerospace and Electronic Systems, 47(1):746-755. https://doi.org/10.1109/TAES.2011.5705706
[The DATE] D. Pastor, F.-X. Socheleau (2012). Robust Estimation of Noise Standard Deviation in Presence of Signals with Unknown Distributions and Occurrences. IEEE Transactions on Signal Processing, 60(4):1545-1555. https://doi.org/10.1109/TSP.2012.2184534
We are quite happy with the DATE. First, as shown in [The DATE], it outperforms standard robust estimators, such as the MAD and standard scale estimators, for detecting noisy noncooperative radio-communications. Second, the DATE and the weak sparseness model, including [A sharp upper bound, Theorem VII.1], to process audio and speech signals. In particular and very importantly, [Noise estimation in speech processing] introduces algorithms based on the DATE to estimate the spectrum of coloured Gaussian noise, whereas the original DATE is dedicated to white Gaussian noise only. As shown in this same paper and in [Weak sparseness in speech denoising] , the performance loss incurred by using DATE-estimators instead of the true noise spectrum is limited in speech denoising.
[Weak sparseness & blind source separation] S. M. Aziz Sbai, A. Aissa El Bey, D. Pastor (2012). Contribution of Statistical Tests to Sparseness-Based Blind Source Separation. Eurasip Journal on Applied Signal Processing, 2012(169). https://doi.org/10.1186/1687-6180-2012-169
[Noise estimation in speech processing] V. K. Mai, D. Pastor, A. Aissa El Bey, R. Le Bidan (2015). Robust Estimation of Non-Stationary Noise Power Spectrum for Speech Enhancement. IEEE Transactions on Audio, Speech and Language Processing, 23(4):670-682. https://doi.org/10.1109/TASLP.2015.2401426
[Weak sparseness in speech denoising] V. K. Mai, D. Pastor, A. Aissa El Bey, R. Le Bidan (2018). Semi-Parametric Joint Detection and Estimation for Speech Enhancement based on Minimum Mean Square Error. Speech Communication, 102:27-38. https://doi.org/10.1016/j.specom.2018.05.005
________________________________________________________________________________
Weak sparseness in wavelet shrinkage
The eagle-eyed reader will have noticed that the weak sparseness model resembles a lot the "strong" sparsity model, as encountered in compressed sensing, for instance. The "strong" sparsity model is that most coefficients representing a signal returned by a sparse transform are null and that only a few of them have large amplitude. This model does not take into account that many signals in the wavelet or the Fourier domains are represented by series of coefficients involving a not so small proportion of large coefficients! As detailed in the papers cited below, weak sparseness turns out to be more suitable to model coefficients pertaining to smooth random signals after sparse transforms than the usual "strong" sparsity model. You'll find examples of spectrograms of audio signals in [Weak sparseness & blind source separation] to illustrate this point in the Fourier domain. Of course, we have the same in the wavelet domain and several examples are given in our series of papers co-autjored with A.M. Atto and G. Mercier and dedicated to wavelet shrinkage based on weak sparseness.
On this topic, I would recommend to read our [Book chapter on wavelet shrinkage & weak sparseness], which synthesizes most of our works by extending our conference paper [Statistics for sparse transforms].
[Book chapter on wavelet shrinkage & weak sparseness] Pastor D. Atto A.M. (2010). Wavelet shrinkage: from sparsity and robust testing to smooth adaptation, In. J. Barral & Seuret (eds): Recent developments in Fractals and Related Fields, Birkhaüser.
[Statistics for sparse transforms] Pastor D., Atto A. M. (2007). Statistics for sparse transforms: some recent results with application to speech and image processing, Fractals and Related Fields: Conference in honor of Jacques Peyrière, 8-13 septembre 2007, Monastir (Tunisie). Proceedings Fractals and Related Fields: Conference in honor of Jacques Peyrière.
For more details in complement to the [Book chapter on wavelet shrinkage & weak sparseness], the interested reader can refer to the following papers.
[Detection thresholds] Atto A., Pastor D., Mercier G. (2008). Detection threshold for non-parametric estimation. Signal, Image and Video Processing, 2(3):207-223. https://doi.org/10.1007/s11760-008-0051-x
[Sigmoid Shrinkage] Atto A., Pastor D., Mercier G. (2009). Smooth Adaptation by Sigmoid Shrinkage. EURASIP Journal on Image and Video Processing, DOI: https://doi.org/10.1155/2009/532312
[Unification] Atto A., Pastor D., Mercier G. (2011). Wavelet Shrinkage: Unification of Basic Thresholding Functions and Thresholds. Signal, Image and Video Processing, 5(1):11-28. DOI: https://doi.org/10.1007/s11760-009-0139-y
In [Detection threshold], we revisit the choice of the thresholds that adjust shrinkage functions to perform signal estimation by separating large from small signal coefficients. We cast the problem into a binary hypothesis testing framework obeying the weak sparseness model and exhibit the detection thresholds that can replace with significant benefit the usual universal and minimax thresholds. The theoretical gain brought by these detection thresholds is experimentally validated in image denoising. In [Sigmoid Shrinkage] and [Unification], we go a step further by revisiting the shrinkage functions themselves. Indeed, smoothness of hard and soft thresholding functions is limited. This holds also for many other thresholding functions proposed in the literature. To bypass such limitation, we introduce a new family of sigmoid-based shrinkage functions. Theoretical benefits of such functions is experimentally verified in image denoising.
To conclude by making a connection with the beginning of the discussion, the funny and interesting fact is that the speech denoising studied in [Weak sparseness in speech denoising] involves the same sigmoid-based shrinkage function as that we introduced to perform wavelet shrinkage. In speech processing too, and thus, in the Fourier domain, this type of shrinkage function performs well. The reason is simple: the weak sparseness model is relevant to model speech Fourier coefficients.
________________________________________________________________________________
Contributions to sparse models in communications
Although our main contributions concern weak sparseness in that it's a concept we introduced ourselves, we also contributed to mathematical problems posed by the use of the strong sparsity model in communications. In this respect, we came up with some interesting theoretical results in the following paper:
Aissa El Bey A., Pastor D., Aziz Sbai S. M., Fadlallah Y. (2015). Sparsity-based Recovery of Finite Alphabet Solutions to Underdetermined Linear Systems. IEEE Transactions on Information Theory, 61(4):2008-2018. DOI: https://doi.org/10.1109/TIT.2015.2399914
and its applications:
Fadlallah Y., Aissa El Bey A., Amis Cavalec K., Pastor D., Pyndiah R. (2015). New Iterative Detector of MIMO Transmission Using Sparse Decomposition. IEEE Transactions on Vehicular Technology, 64(8):3458-3464. DOI: https://doi.org/10.1109/TVT.2014.2360687
________________________________________________________________________________
Appendix: conference papers
The following conference papers can profitably be used by the reader to further analyze our approach on weak sparseness and sparse transforms in communications. In particular, these papers give verious examples of applications.
Mai V. K., Pastor D., Aissa El Bey A. (2018). Block Smoothed Sigmoid-Based Shrinkage in Time-Frequency Domain for Robust Audio Denoising, 9th International Symposium on Signal, Image, Video and Communications (ISIVC), 27-30 novembre 2018, Rabat (Maroc). Proceedings 9th International Symposium on Signal, Image, Video and Communications (ISIVC).
Mai V. K., Pastor D., Aissa El Bey A. (2018). Joint Soft Threshold and Statistical Estimation for Speech Enhancement, 9th International Symposium on Signal, Image, Video and Communications (ISIVC), 27-30 November 2018, Rabat (Maroc). Proceedings 9th International Symposium on Signal, Image, Video and Communications (ISIVC).
Mai V. K., Pastor D., Aissa El Bey A., Le Bidan R. (2017). Combined Detection and Estimation Based on Mean-Square Error Log-Spectral Amplitude for Speech Enhancement, GRETSI 2017: 26ème colloque du Groupement de Recherche en Traitement du Signal et des Images, 5-8 septembre 2017, Juan-Les-Pins (France). Proceedings GRETSI 2017: 26ème colloque du Groupement de Recherche en Traitement du Signal et des Images.
Aissa El Bey A., Pastor D. (2015). Reconstruction par transformation parcimonieuse de solutions à alphabet fini de systèmes linéaires sous-déterminés, GRETSI 2015: 25ème colloque du Groupement de Recherche en Traitement du Signal et des Images, 8-11 septembre 2015, Lyon (France).
Fadlallah Y., Aissa El Bey A., Amis Cavalec K., Pastor D. (2015). Low-complexity detector for very large and massive MIMO transmission, SPAWC 2015: 16th IEEE International Workshop on Signal Processing Advances in Wireless Communications, 28 juin-1 juillet 2015, Stockholm (Suède). 251 - 255. DOI: https://doi.org/10.1109/SPAWC.2015.7227038
Pastor D., Aissa El Bey A. (2015). Discrete solutions to underdetermined linear systems via sparse-based transform, SPARS 2015: Signal Processing with Adaptive Sparse Structured Representations Workshop, 6-9 juillet 2015, Cambridge (Royaume-Uni). Proceedings SPARS 2015: Signal Processing with Adaptive Sparse Structured Representations Workshop.
Fadlallah Y., Aissa El Bey A., Amis Cavalec K., Pastor D., Pyndiah R. (2013). New decoding strategy for underdetermined mimo transmission sparse decomposition, EUSIPCO 2013: 21st European Signal Processing Conference, 9-13 septembre 2013, Marrakech (Maroc). Proceedings EUSIPCO 2013: 21st European Signal Processing Conference.
Aziz Sbai S. M., Aissa El Bey A., Pastor D. (2012). Underdetermined Source Separation of Finite Alphabet Signals Via L1 Minimization, ISSPA 2012: 11th IEEE International Conference on Information Sciences, Signal Processing and their Applications, 3-5 juillet 2012, Montreal, Quebec (Canada). Proceedings ISSPA 2012: 11th IEEE International Conference on Information Sciences, Signal Processing and their Applications, 625 - 628.
Aziz Sbai S. M., Aissa El Bey A., Pastor D. (2011). Recovery of finite alphabet signals from incomplete measurements, SPARS, 27-30 juin 2011, Edimbourg (Royaume-Uni). Proceedings SPARS.
Aziz Sbai S. M., Aissa El Bey A., Pastor D. (2011). Robust underdetermined blind audio source separation of sparse signals in the time-frequency domain, ICASSP 2011: 36th IEEE International Conference on Acoustics, Speech, and Signal Processing, 22-27 mai 2011, Prague (République Tchèque). Proceedings ICASSP 2011: 36th IEEE International Conference on Acoustics, Speech, and Signal Processing.
Pastor D., Atto A. (2011). Sparseness-based non-parametric detection and estimation of random signals in noise, Spars, 27-30 juin 2011, Edimbourg (Royaume-Uni). Proceedings Spars.
Atto A. M., Mercier G., Pastor D. (2009). General framework on change detection in a sparse domain, IGARSS 2009: IEEE International Geoscience & remote Sensing Symposium, 12-17 juillet 2009, Cape Town (Afrique du Sud). Proceedings IGARSS 2009: IEEE International Geoscience & remote Sensing Symposium.
Atto A. M., Pastor D., Mercier G. (2009). Optimal sure parameters for sigmoidal wavelet shrinkage, EUSIPCO 2009: 17th European Signal Processing Conference, 24-28 août 2009, Glasgow (Royaume-Uni). Proceedings EUSIPCO 2009: 17th European Signal Processing Conference.
Atto A., Pastor D., Mercier G. (2009). Sparsity Measure and the Detection of Significant Data, SPARS'09 - Signal Processing with Adaptive Sparse Structured Representations, 6 avril 2009, Saint Malo (France).
Pastor D., Socheleau F.-X., Aissa El Bey A. (2009). Sparsity hypotheses for robust estimation of the noise standard deviation in various signal processing applications, SPARS'09 - Signal Processing with Adaptive Sparse Structured Representations, 6 avril 2009, Saint Malo (France).
Socheleau F.-X., Pastor D., Aissa El Bey A., Houcke S. (2009). Blind noise variance estimation for OFDMA signals, ICASSP 2009: 34th IEEE International Conference on Acoustics, Speech, and Signal Processing, 19-24 avril 2009, Taipei (Taïwan). Proceedings ICASSP 2009: 34th IEEE International Conference on Acoustics, Speech, and Signal Processing, 2581 - 2584. DOI: https://doi.org/10.1109/ICASSP.2009.4960150
Atto A. M., Pastor D., Mercier G. (2008). Smooth sigmoid wavelet shrinkage for non-parametric estimation, ICASSP 2008: IEEE international conference on acoustics, speech and signal processing, March 30 - April 4, Las Vegas, USA, 30 mars-4 avril 2008, Las Vegas (États-Unis). Proceedings ICASSP 2008: IEEE international conference on acoustics, speech and signal processing, March 30 - April 4, Las Vegas, USA, 3265 - 3268. DOI: https://doi.org/10.1109/ICASSP.2008.451834
Pastor D., Atto A. M. (2007). Statistics for sparse transforms: some recent results with application to speech and image processing, Fractals and Related Fields: Conference in honor of Jacques Peyrière, 8-13 septembre 2007, Monastir (Tunisie). Proceedings Fractals and Related Fields: Conference in honor of Jacques Peyrière.
Pastor D. (2006). Estimating the standard deviation of some additive white Gaussian noise on the basis of non signal-free observations, ICASSP'06: IEEE International Conferece on Acoustics, Speech and Signal Processing, 14-19 mai 2006, Toulouse (France). Proceedings ICASSP'06: IEEE International Conferece on Acoustics, Speech and Signal Processing, 3. DOI: https://doi.org/10.1109/ICASSP.2006.1660737
Pastor D. (2005). Two results in statistical decision theory for detecting signals with unknown distributions and priors in white Gaussian noise, AMSDA'05: International Symposium on Applied Stochastic Models and Data Analysis, 17-20 mai 2005, Brest (France). Proceedings AMSDA'05: International Symposium on Applied Stochastic Models and Data Analysis.
Pastor D. (2005). Un théorème limite et un test pour la détection non paramétrique de signaux dans un bruit blanc Gaussien de variance inconnue, GRETSI'05: 20ème colloque sur le traitement du signal et des images, 6-9 septembre 2005, Louvain La Neuve (Belgique). Actes GRETSI'05: 20ème colloque sur le traitement du signal et des images.