Deformable Spectrograms

Manuel Reyes-Gomez; Nebojsa Jojic; Daniel P. W. Ellis

Deformable Spectrograms
Reyes-Gomez, Manuel
Jojic, Nebojsa
Ellis, Daniel P. W.
Electrical Engineering
Persistent URL:
Book/Journal Title:
AISTATS 2005: Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, January 6-8, 2005, The Savannah Hotel, Barbados
Society for Artificial Intelligence and Statistics
Speech and other natural sounds show high temporal correlation and smooth spectral evolution punctuated by a few, irregular and abrupt changes. In a conventional Hidden Markov Model (HMM), such structure is represented weakly and indirectly through transitions between explicit states representing 'steps' along such smooth changes. It would be more efficient and informative to model successive spectra as transformations of their immediate predecessors, and we present a model which focuses on local deformations of adjacent bins in a time-frequency surface to explain an observed sound, using explicit representation only for those bins that cannot be predicted from their context. We further decompose the log-spectrum into two additive layers, which are able to separately explain and model the evolution of the harmonic excitation, and formant filtering of speech and similar sounds. Smooth deformations are modeled with hidden transformation variables in both layers, using Markov Random Fields (MRFs) with overlapping subwindows as observations; inference is efficiently performed via loopy belief propagation. The model can fill-in deleted time-frequency cells without any signal model, and an entire signal can be compactly represented with a few specific states along with the deformation maps for both layers. We discuss several possible applications for this new model, including source separation.
Electrical engineering
Item views
text | xml
Suggested Citation:
Manuel Reyes-Gomez, Nebojsa Jojic, Daniel P. W. Ellis, , Deformable Spectrograms, Columbia University Academic Commons, .

Columbia University Libraries | Policies | FAQ