{"id":1240,"date":"2023-05-22T11:25:05","date_gmt":"2023-05-22T11:25:05","guid":{"rendered":"https:\/\/catedramasmovil.uc3m.es\/2023\/05\/22\/emotion-transfer-in-voice-using-neural-networks\/"},"modified":"2023-05-22T13:55:20","modified_gmt":"2023-05-22T13:55:20","slug":"emotion-transfer-in-voice-using-neural-networks","status":"publish","type":"post","link":"https:\/\/catedramasmovil.uc3m.es\/en\/2023\/05\/22\/emotion-transfer-in-voice-using-neural-networks\/","title":{"rendered":"Emotion transfer in voice using neural networks…"},"content":{"rendered":"

[et_pb_section fb_built=”1″ _builder_version=”4.17.0″ custom_padding=”0px||||false|false” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_row _builder_version=”4.17.0″ _module_preset=”default” custom_padding=”0px||||false|false” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_column type=”4_4″ _builder_version=”4.17.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_gallery gallery_ids=”1177,1181″ fullwidth=”on” _builder_version=”4.21.0″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”et_body_layout” sticky_enabled=”0″][\/et_pb_gallery][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.16″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_column type=”4_4″ _builder_version=”4.16″ custom_padding=”|||” global_colors_info=”{}” custom_padding__hover=”|||” theme_builder_area=”et_body_layout”][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”]<\/p>\n

Objective<\/h2>\n

The objective of this project is to develop a machine learning model to transform the emotion expressed in the voice. Emotion transfer is an open problem, and significant progress has been made thanks to the emergence of generative adversarial networks.<\/p>\n

In this work, we propose a model that builds upon the work of [1], based on a specific type of generative adversarial networks called CycleGAN, and we incorporate some architectural updates that enhance the quality of the synthesized voices. We address three issues:<\/p>\n

    \n
  1. Traditionally, models have been trained on parallel data, where there are examples of utterances spoken with different emotions but sharing linguistic content. This limitation made it impossible to use real data for training and restricted the usefulness of the models. CycleGAN networks can leverage real-world data because they do not require parallel data.<\/li>\n
  2. Emotion in speech is mostly related to prosodic aspects of the voice, such as pitch or rhythm. To achieve effective emotion transfer it is necessary to transform these features. In this project we decompose the fundamental frequency of the voice with continuous wavelet transform, which has shown to improve the conversion of the fundamental frequency, and consequently, of prosody.<\/li>\n
  3. The quality of the transformed voices is inferior to the original voices. In this project, we incorporate an updated CycleGAN architecture proposed by [2] for identity transfer problems, and show that the quality of the synthesized voices improves compared to the baseline model.<\/li>\n<\/ol>\n

    The final model enhances spectrum transformation, prosody, and overall quality of the synthesized voices.<\/p>\n

     <\/p>\n

    [1] K. Zhou, B. Sisman y H. Li, Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data, en, arXiv:2002.00198 [cs,eess], oct. de 2020. [En l\u00ednea]. Disponible en: http:\/\/arxiv.org\/abs\/2002.00198 (Acceso: 04-04-2023).<\/p>\n

    [2] T. Kaneko, H. Kameoka, K. Tanaka y N. Hojo, CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion, en, arXiv:1904.04631 [cs, eess, stat], abr. de 2019. [En l\u00ednea]. Disponible en: http:\/\/arxiv.org\/abs\/1904.04631 (Acceso: 04-04-2023).<\/p>\n

    [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.18.0″ _module_preset=”default” custom_margin=”20px||||false|false” custom_padding=”0px||||false|false” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_row column_structure=”1_2,1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_column type=”1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_image src=”https:\/\/storage.googleapis.com\/wp-uploads.bucket.wp.uc3m.es\/wp-content\/uploads\/sites\/70\/2023\/05\/22113212\/foto_perfil_pablo-1-scaled.jpg” title_text=”foto_perfil_pablo” _builder_version=”4.21.0″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”et_body_layout” sticky_enabled=”0″][\/et_pb_image][\/et_pb_column][et_pb_column type=”1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_text _builder_version=”4.21.0″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”et_body_layout” sticky_enabled=”0″]<\/p>\n

    BACHELOR’S THESIS BY:<\/strong><\/span><\/p>\n

    PABLO D\u00cdAZ LARRA\u00cdN
    <\/strong><\/span><\/p>\n

    [\/et_pb_text][et_pb_text _builder_version=”4.20.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”]<\/p>\n

    Degree<\/strong><\/p>\n

    Degree in Computer Science and Engineering<\/p>\n

     <\/p>\n

    Work Experience<\/strong><\/p>\n

    Researcher at C\u00e1tedra UC3M-M\u00e1sM\u00f3vil (September 2022 – May 2023)<\/p>\n

     <\/p>\n

    Technical skills<\/strong><\/p>\n

    Programming languages: Python, C\/C++, Javascript.<\/p>\n

    Libraries: Tensorflow, NumPy, Pandas.<\/p>\n

    Platforms: Google Cloud Platform y Vertex AI.<\/p>\n

     <\/p>\n

    [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"

    [et_pb_section fb_built=”1″ _builder_version=”4.17.0″ custom_padding=”0px||||false|false” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_row _builder_version=”4.17.0″ _module_preset=”default” custom_padding=”0px||||false|false” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_column type=”4_4″ _builder_version=”4.17.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_gallery gallery_ids=”1177,1181″ fullwidth=”on” _builder_version=”4.21.0″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”et_body_layout” sticky_enabled=”0″][\/et_pb_gallery][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.16″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” global_colors_info=”{}” theme_builder_area=”et_body_layout”][et_pb_column type=”4_4″ _builder_version=”4.16″ custom_padding=”|||” global_colors_info=”{}” custom_padding__hover=”|||” theme_builder_area=”et_body_layout”][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”et_body_layout”] Objective The objective of this project is to develop a machine learning model to transform […]<\/p>\n","protected":false},"author":172,"featured_media":1178,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[59],"tags":[],"class_list":["post-1240","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-projects-2022-2023"],"_links":{"self":[{"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/posts\/1240","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/users\/172"}],"replies":[{"embeddable":true,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/comments?post=1240"}],"version-history":[{"count":4,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/posts\/1240\/revisions"}],"predecessor-version":[{"id":1246,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/posts\/1240\/revisions\/1246"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/media\/1178"}],"wp:attachment":[{"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/media?parent=1240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/categories?post=1240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/catedramasmovil.uc3m.es\/en\/wp-json\/wp\/v2\/tags?post=1240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}