MTU - Publications

Highlights

Project page Code

2026

CrowdioSet and PaRIRset: Two Datasets Towards Live Music Source Separation

Guso, E. & Serra, X.

We introduce CrowdioSet, a noise dataset of real crowd ambiences and synthetic sing-alongs for training music source separation models on live recordings, and PaRIRset, a stereo room impulse response dataset captured with a microphone array across 40 professional concert venues with PA systems. Together they improve music source separation on live recordings over training on studio data alone, in both objective and subjective evaluations.

Proceedings of the 27th International Society for Music Information Retrieval Conference (ISMIR 2026)

Project page Code

2026

ShinyNeRF: Digitizing Anisotropic Appearance in Neural Radiance Fields

Barreiro, A., Marí, R., Redondo, R., Haro, G. & Bosch, C.

A physically-based NeRF variant that jointly estimates geometry and anisotropic specular parameters from multi-view images.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (pp. 33-40), 2026.

✨ Best paper award at 3D-ARCH 2026

Code

2026

MExECON: Multi-View Extended Explicit Clothed Humans Optimized via Normal Integration

Ece Ugur, F., Redondo, R., Barreiro, A., Hristov, S. & Marí, R.

MExECON reconstructs clothed human avatars from sparse RGB images. It extends the single-view ECON method to an arbitrary number of viewpoints, improving geometry and pose estimation without retraining.

International Conference on Computer Vision Theory and Applications (VISAPP 2026).

Code

2025

MB-RIRs: a Synthetic Room Impulse Response Dataset with Frequency-Dependent Absorption Coefficients

Guso, E., Luberadzka, J., Sayin, U. & Serra, X.

We open-source a synthetic reverberation dataset that improves the performance of data-driven speech models. We show that Multi-Band (MB) absorption coefficients on shoebox-based synthetic RIRs lead to better performance in DNN-based speech enhancement compared to using single-band (SB) absorption coefficients. We additionally study implementing source and receiver directivities.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2025)

Project page Code

2025

Conditioned Wave-U-Net for Acoustic Matching of Speech in Shared XR Environments

Luberadzka, J., Guso, E. & Sayin, U.

A model for acoustic style transfer in speech. The reverberation (r1) of a speech signal (s1) is transformed into s1r2 by Film-conditioning a U-net with another reverberant speech signal s2r2. Our model outperforms two baselines and performs on par with a semi-oracle case.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2025)

2025

Emotional Impact of Source Localization in Music Using Machine Learning and EEG

Schmele, T., De Filippi, E., Nandi, A., Pereda-Banos, A. & Garriga, A.

Machine learning methods were optimized on spectral EGG features according to four music source positions and valence type in a subject-wise manner, finding that EEG correlates with four source locations, specially when a negative emotional valence is played outside of the listener's field of view.

Music and Sound Generation in the AI Era, Springer Nature, 2023.

2025

A Study on Interpretation of Medieval Vocal Repertoire through Auralisation: An Acoustic Reconstruction of the Great Chapel of the Palais des Papes

De Muynke, J., Ferrando, J., & Katz, B.F.G.

The study reconstructs the acoustics of the Great Chapel at the Palais des Papes to examine how its changing architecture influenced 14th-century Ars Nova choral performance. Through real-time virtual auralisation, singers experienced historically accurate acoustics, offering new insights into the interplay between medieval music and its performance spaces.

Journal of Sound Studies, 2025.

2024

Effect of interpolation artifacts on perceived stability of nearby sources in a navigable reverberant virtual environment

De Muynke, J., Poirier-Quinot, D., & Katz, B.F.G.

The study explores how interpolation between spatial Room Impulse Responses affects the perceived stability of sound sources in navigable reverberant virtual environments. Using third-order Ambisonic RIR convolution with MagLS binaural decoding, VR listening tests inform the design of realistic spatial audio for immersive experiences such as virtual museum visits.

Journal of the Audio Engineering Society, 2024.

2024

Spatial Matrix Synthesis

Schmele, T., & Garriga, A.

Definition of a method to synthesize new sounds using only spatial transformation matrices in 2D and 3D audio space.

Journal of the Audio Engineering Society, 72(4), 193-210, 2024.

2023

Listen and Move: Improving GANs Coherency in Agnostic Sound-to-Video Generation

Redondo, R.

GANs architectural improvements for temporal coherency on generic sound-to-video generation, including triple sound routing, a multiscale dilated RNN, and a novel recurrent and directional video prediction layer.

AV4D: Visual Learning of Sounds in Spaces. ICCV Workshop, 2023.

Code

2023

An Objective Evaluation of Hearing AIDS and DNN-Based Binaural Speech Enhancement in Complex Acoustic Scenes

Gusó, E., Luberadzka, J., Baig, M., Sayin, U., & Serra, X.

The performance of five high-end commercially available Hearing Aid (HA) devices are compared to DNN-based speech enhancement algorithms in complex acoustic environments, in which the latter outperforms the HA algorithms in terms of noise suppression and objective intelligibility metrics.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (pp. 1-5), 2023.

Full List

2026

MExECON: Multi-View Extended Explicit Clothed Humans Optimized via Normal Integration

Ece Ugur, F., Redondo, R., Barreiro, A., Hristov, S. & Marí, R.

International Conference on Computer Vision Theory and Applications (VISAPP 2026).

Code

Computer Vision 3D Reconstruction Avatars
ShinyNeRF: Digitizing Anisotropic Appearance in Neural Radiance Fields

Barreiro, A., Marí, R., Redondo, R., Haro, G. & Bosch, C.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (pp. 33-40), 2026.

Project page Code

✨ Best paper award at 3D-ARCH 2026

Computer Vision Neural Rendering 3D Reconstruction
CrowdioSet and PaRIRset: Two Datasets Towards Live Music Source Separation

Guso, E. & Serra, X.

Proceedings of the 27th International Society for Music Information Retrieval Conference (ISMIR 2026)

Project page Code

Audio and Acoustics Datasets and Benchmarks
An AI Voice Agent for Speech Communication Assessment

Luberadzka, J. & Sayin, U.

Proceedings of the 7th Virtual Conference on Computational Audiology (VCCA)

Code Presentation

✨ Best Podium Presentation award at VCCA 2026

AI Voice Agents Speech and Hearing

2025

Reconstructing acoustic heritage: Methods and tools for interdisciplinary research and audio-augmented experiences

De Muynke, J.

PhD dissertation, Sorbonne Université, 2025.

Acoustic Heritage and Auralisation Cultural Heritage Audio and Acoustics
Latent Diffusion Approaches for Conditional Generation of Aerial Imagery: A Study

Marí, R. and Redondo, R.

Image Processing On Line, 2025.

Computer Vision Generative AI and Diffusion Models Remote Sensing and Earth Observation
Specularity in NeRFs: A Comparative Study of Ref-NeRF and NRFF

Barreiro, A. and Marí, R. and Redondo, R. and Haro, G. and Bosch, C. and Berga, D.

Image Processing On Line, 2025.

Computer Vision Neural Rendering 3D Reconstruction
Audio technology for improving social interaction in extended reality

J. Luberadzka, E. Gusó Muñoz, U. Sayin, and A. Garriga

Front. Virtual Real., Sec. Technologies for VR, Volume 5 - 2024, 23 January 2025.

XR and Immersive Media Speech and Hearing Audio and Acoustics
A Study on Interpretation of Medieval Vocal Repertoire through Auralisation: An Acoustic Reconstruction of the Great Chapel of the Palais des Papes

De Muynke, J., Ferrando, J., & Katz, B.F.G.

Journal of Sound Studies, 2025.

Acoustic Heritage and Auralisation Cultural Heritage Audio and Acoustics
Using Virtual Reality to Raise Awareness of Communication Challenges Faced by Individuals with Hearing Loss

Luberadzka, J., Gusó, E., Sayin, U., & Garriga, A.

IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 1640-1641), 2025.

XR and Immersive Media Speech and Hearing Human-Computer Interaction
S-EO: A Large-Scale Dataset for Geometry-Aware Shadow Detection in Remote Sensing Applications

Masquil, E., Marí, R., Ehret, T., Meinhardt-Llopis, E., Musé, P., & Facciolo, G.

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2025).

Computer Vision Remote Sensing and Earth Observation Datasets and Benchmarks
Emotional Impact of Source Localization in Music Using Machine Learning and EEG

Schmele, T., De Filippi, E., Nandi, A., Pereda-Banos, A. & Garriga, A.

Music and Sound Generation in the AI Era, Springer Nature, 2023.

Audio and Acoustics Affective Computing and EEG
Differentiating Sensations of Sound Envelopment in Spatial Sound Synthesis Approaches: an Explorative Study

Schmele, T. & De Filippi, E.

159th Convention of the Audio Engineering Society, Long Beach, CA, USA, October 23-25, 2025.

Audio and Acoustics Spatial Audio
Conditioned Wave-U-Net for Acoustic Matching of Speech in Shared XR Environments

Luberadzka, J., Guso, E. & Sayin, U.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2025)

Project page Code

Audio and Acoustics Speech and Hearing XR and Immersive Media
MB-RIRs: a Synthetic Room Impulse Response Dataset with Frequency-Dependent Absorption Coefficients

Guso, E., Luberadzka, J., Sayin, U. & Serra, X.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2025)

Code

Audio and Acoustics Speech and Hearing Datasets and Benchmarks
LAND: Lung and Nodule Diffusion for 3D Chest CT Synthesis with Anatomical Guidance

Oliveras, A., Marí, R., Redondo, R., Guardià, O., Tost, A., Nagarajan, B., Migliorelli, C., Ribas, V., & Radeva, P.

Medical Imaging meets EurIPS: MedEurIPS 2025 (EurIPS Workshops)

Medical Imaging Generative AI and Diffusion Models

2024

Spatial Matrix Synthesis

Schmele, T., & Garriga, A.

Journal of the Audio Engineering Society, 72(4), 193-210, 2024.

Audio and Acoustics Spatial Audio
HeSoAN: un projet archéoacoustique

Ferrando, J., De Muynke, J., Katz, B.F.G.

Résonances Gothiques, Collegium Musicae, 2024.

Acoustic Heritage and Auralisation Cultural Heritage
Interpretation of a medieval vocal repertoire in the reconstructed acoustics of the Great Chapel of the Palais des Papes

Ferrando, J. & De Muynke, J.

Historical Resonances: Space, Senses and Early Music, Brill Fink, The Netherlands, 2024.

Acoustic Heritage and Auralisation Cultural Heritage Audio and Acoustics
The Past Has Ears at Notre-Dame: Acoustic digital twins for research and narration

Katz, B.F.G., Cros C., Peichert, S., De Muynke, J.

Digital Applications in Archaeology and Cultural Heritage. 2024.

Acoustic Heritage and Auralisation Cultural Heritage Visualization and Digital Twins
Characterization of Synthetic Lung Nodules in Conditional Latent Diffusion of Chest CT Scans

Marí Molas, R., Subías-Beltrán, P., Pitarch Abaigar, C., Galofré Cardo, M., & Redondo Tejedor, R.

In Artificial Intelligence Research and Development (pp. 44-51). IOS Press, 2024.

Code

Medical Imaging Generative AI and Diffusion Models
Whispers of the Past at Notre-Dame: a geo-localised immersive walk through the sonic memory of Notre-Dame de Paris cathedral

Peichert, S., De Muynke, J., & Katz, B.F.G

Résonances Gothiques, Collegium Musicae, 2024.

Acoustic Heritage and Auralisation Cultural Heritage XR and Immersive Media
Effect of interpolation artifacts on perceived stability of nearby sources in a navigable reverberant virtual environment

De Muynke, J., Poirier-Quinot, D., & Katz, B.F.G.

Journal of the Audio Engineering Society, 2024.

Audio and Acoustics Spatial Audio XR and Immersive Media
Schmele-Spherical Glitch Study

Schmele, T.

In Speculative Sound Synthesis Symposium, 2024.

Audio and Acoustics Spatial Audio

2023

Sound spectrum modulation generated by circularly moving sound sources

Schmele, T., & Garriga, A.

Audio Engineering Society Convention 154, 2023.

Audio and Acoustics Spatial Audio
An Objective Evaluation of Hearing AIDS and DNN-Based Binaural Speech Enhancement in Complex Acoustic Scenes

Gusó, E., Luberadzka, J., Baig, M., Sayin, U., & Serra, X.

IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (pp. 1-5), 2023.

Code

Audio and Acoustics Speech and Hearing
Creating navigable auralisations using RIR convolution: Impact of grid density and panning method on perceived source stability

De Muynke, J., Poirier-Quinot, D., & Katz, B.F.G.

Proceedings of the AES Conference on Spatial and Immersive Audio, 2023.

Audio and Acoustics Spatial Audio XR and Immersive Media
Listen and Move: Improving GANs Coherency in Agnostic Sound-to-Video Generation

Redondo, R.

AV4D: Visual Learning of Sounds in Spaces. ICCV Workshop, 2023.

Generative AI and Diffusion Models Multimodal AI
Comparison of synthesized Virtual Sound Environments with validated Hearing Aid experiments

Sayin Saraç, U.

In Audio Engineering Society Convention 155. Audio Engineering Society, 2023.

Audio and Acoustics Spatial Audio Speech and Hearing
Emulating Vector Base Amplitude Panning Using Panningtable Synthesis

Schmele, T., & Reppel, N.

In Audio Engineering Society Convention 155. Audio Engineering Society, 2023.

Audio and Acoustics Spatial Audio
Emotional Impact of Source Localization in Music Using Machine Learning and EEG: a proof-of-concept study

De Filippi, E., Schmele, T., Nandi, A., Torres, A. G., & Pereda-Banos, A.

Authorea Preprints, 2023.

Audio and Acoustics Affective Computing and EEG Multimodal AI
A New Framework for Evaluating Image Quality Including Deep Learning Task Performances as a Proxy

Gallés, P., Takáts, K., Hernández-Cabronero, M., Berga, D., Pega, L., Riordan-Chen, L., ... & Marín, J.

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 3285-3296, 2023.

Computer Vision Remote Sensing and Earth Observation Datasets and Benchmarks

2022

Ears of the past, an inquiry into the sonic memory of the acoustics of Notre-Dame before the fire of 2019

De Muynke, J., Baltazar, M., Monferran, M., Voisenat, C., & Katz, B.F.G.

Journal of Cultural Heritage, vol. 56, pp. 130--137, 2022.

Acoustic Heritage and Auralisation Cultural Heritage
Spatially Oriented Format for Acoustics 2.1: Introduction and Recent Advances

Majdak, P., Zotter, F., Brinkmann, F., De Muynke, J., Mihocic, M., & Noisternig, M.

Journal of the Audio Engineering Society, vol. 70, no 7/8, pp. 565--584, 2022.

Audio and Acoustics Spatial Audio
Preliminary analysis of vocal ensemble performances in real-time historical auralizations of the Palais des Papes

De Muynke, J., Eley, N., Ferrando, J., & Katz, B.F.G.

Proceedings of the 2nd Symposium; The Acoustics of Ancient Theaters, 6-8 July 2022, Verona, Italy

Acoustic Heritage and Auralisation Cultural Heritage Audio and Acoustics
On loss functions and evaluation metrics for music source separation.

Gusó, E., Pons, J., Pascual, S., & Serrà, J.

In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 306-310).

Audio and Acoustics
Use of synthesized higher order ambisonics impulse responses to generate virtual sound environments for hearing aids research

Sayin Sarac, U.

Journal of Hearing Science, 12(1), 2022.

Audio and Acoustics Spatial Audio Speech and Hearing
Comparisons between VBAP and WFS using Spatial Sound Synthesis

Schmele, T., & Lopez, J. J.

In Audio Engineering Society Convention 153. Audio Engineering Society, 2022.

Audio and Acoustics Spatial Audio
IQUAFLOW: A new framework to measure image quality.

Gallés, P., Takats, K., Hernández-Cabronero, M., Berga, D., Pega, L., Riordan-Chen, L., ... & Marín, J.

arXiv preprint arXiv:2210.13269.

Computer Vision Datasets and Benchmarks

2020

ARSI: An Aerial Robot for Sewer Inspection

F. Chataigner, P. Cavestany, M. Soler, C. R., J. P. Gonzalez, C. Bosch, J. Gibert, A. Torrente, R. Gomez and D. Serrano.

Advances in Robotics Research: From Lab to Market, Springer, 2020.

Computer Vision 3D Reconstruction Graphics and Simulation
Exploring Hate Speech Detection in Multimodal Publications

Gomez, R., Gibert, J., Gomez, L., & Karatzas, D.

arXiv preprint arXiv:1910.03814, 2020.

Multimodal AI Computer Vision

2019

Selective Style Transfer for Text

Gomez, R., Biten, A.F., Gomez, L., Gibert, J., Rusiñol, M., & Karatzas, D.

IEEE International Conference on Document Analysis and Recognition (ICDAR) (pp. 805-812), 2019.

Computer Vision Multimodal AI
Controllable Image-Based Transfer of Flow Phenomena

Bosch, C., & Patow, G.

Computer Graphics Forum (Vol. 38, No. 1, pp. 274-285), Feb 2019.

Computer Vision Graphics and Simulation
Self-Supervised Learning from Web Data for Multimodal Retrieval

Gomez, R., Gomez, L., Gibert, J., & Karatzas, D.

Multimodal Scene Understanding (pp. 279-306). Academic Press, 2019.

Computer Vision Multimodal AI
A hybrid parametric-deep learning approach for sound event localization and detection

Pérez-López, A., Fonseca, E., & Serra, X.

ArXiv preprint arXiv:1908.10133, 2019.

Audio and Acoustics Spatial Audio
Analysis of spherical isotropic noise fields with an A-Format tetrahedral microphone

Perez-Lopez, A., & Stefanakis, N.

The Journal of the Acoustical Society of America, 146(4), EL329-EL334, 2019.

Audio and Acoustics Spatial Audio
Artificial Intelligence in the Media and Creative Industries

G. Amato, M. Behrmann, F. Bimbot, B. Caramiaux, F. Falchi, A. Garcia, J. Geurts, J. Gibert, G. Gravier, H. Holken, H. Koenitz, S. Lefebvre, A. Liutkus, F. Lotte, A. Perkis, R. Redondo, E. Turrin, T. Viéville, E. Vincent.

New European Media (NEM), White paper, April 2019.

Multimodal AI
Responsive geographical information systems for spatio-temporal analysis of mobile networks in Barcelona

Sacramento Gutierres, F. R., Torrente, A., & Torrent-Moreno, M.

ACE: architecture, city and environment, 14(40), 163-192.

Visualization and Digital Twins

2018

Tools for the production of spatial audio within BINCI

Kruh-Elendt, A., Fiebig, A., Sottek, R., & De Muynke, J.

DAGA 2018.

Audio and Acoustics Spatial Audio XR and Immersive Media
Ambisonics Directional Room Impulse Response as a new Convention of the Spatially Oriented Format for Acoustics

Pérez-López, A., & De Muynke, J.

AES 144th Convention, May 2018.

Audio and Acoustics Spatial Audio
Ambiscaper: A Tool for Automatic Generation and Annotation of Reverberant Ambisonics Sound Scenes

Perez-Lopez, A.

16th IEEE International Workshop on Acoustic Signal Enhancement (IWAENC) (pp. 1-9), September 2018.

Audio and Acoustics Spatial Audio Datasets and Benchmarks
Learning to learn from web data through deep semantic embeddings

Gomez, R., Gomez, L., Gibert, J., & Karatzas, D.

Proceedings of the European Conference on Computer Vision (ECCV), 2018.

Computer Vision Multimodal AI
Sound-Space Choreography

Erruz, G., & Schmele, T.

Audio Engineering Society Conference: 2018 AES International Conference on Spatial Reproduction-Aesthetics and Science. Audio Engineering Society. July 2018.

Spatial Audio XR and Immersive Media
Urban weathering: Interactive rendering of polluted cities

Muñoz-Pandiella, I., Bosch, C., Mérillou, N., Patow, G., Mérillou, S., & Pueyo, X.

IEEE transactions on visualization and computer graphics, 24(12), 3239-3252, 2018.

Graphics and Simulation Visualization and Digital Twins
Learning from #Barcelona Instagram data what Locals and Tourists post about its Neighbourhoods

Gomez, R., Gomez, L., Gibert, J., & Karatzas, D.

Proceedings of the European Conference on Computer Vision (ECCV), 2018.

Computer Vision Multimodal AI
Psychophysiological Methods for Quality of Experience Research in Virtual Reality Systems and Applications

M. Barreda-Ángeles, R. Redondo-Tejedor, A. Pereda-Baños.

MMTC Communications - Frontiers (IEEE), Vol. 13, No. 1, January 2018.

XR and Immersive Media Affective Computing and EEG Human-Computer Interaction
Controlling the apparent sourcesize in ambisonics using decorrelation filters

Schmele, T., & Sayin, U.

AES International Conference on Spatial Reproduction-Aesthetics and Science. Audio Engineering Society, 2018.

Audio and Acoustics Spatial Audio

2017

Real‐Time Solar Exposure Simulation in Complex Cities

Muñoz‐Pandiella, I., Bosch, C., Mérillou, N., Pueyo, X., & Mérillou, S.

Computer Graphics Forum (Vol. 36, No. 8, pp. 554-566), December 2017.

Graphics and Simulation Visualization and Digital Twins
Towards Semi-Automatic Scaling Detection on Flat Stones

Muñoz-Pandiella, I., Akoglu, K., Bosch, C., & Rushmeier, H. E.

GHC (pp. 49-58), September 2017.

Computer Vision Cultural Heritage
Binaural tools for 3D audio production at home

Garriga, A., Fuenmayor, M. E., & Caballero, M.

Proc. NEM Summit, 2017.

Audio and Acoustics Spatial Audio
Seamless 3D interaction of virtual and real objects in professional virtual studios

Hach, T., Arias, P., Bosch, C., Montesa, J., & Gascó, P.

SMPTE Motion Imaging Journal, 126(1), 43-56, 2017.

XR and Immersive Media Human-Computer Interaction