A media-mashup engine with interactive cross-media association and personalization mechanisms

This paper proposes a media-mashup engine with interactive cross-media association mechanisms that compute correlation values between image data and music data. The aim of this study is creating a database function for decorating a multimedia data with other types of multimedia data by inspecting the contents of a multimedia data. This media mashup engine has several feature vector spaces for computing correlation between heterogeneous media data. This system utilizes a feature transmission matrix that defines relationship between a feature in a vector space and a feature in another vector space. The important feature of this system is a personalization mechanism for this feature transmission matrix, which reflects individual perceptual preferences for defining the cross-media feature associations. The system provides a real-time feedback mechanism to define a user's individual preference by submitting an existing media data. This paper shows a prototype system that implements the cross-media association functions and personalization mechanisms. This paper shows several experimental results to clarify the effectiveness and feasibility of this system.