{"id":1266,"date":"2023-06-14T12:53:36","date_gmt":"2023-06-14T10:53:36","guid":{"rendered":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/"},"modified":"2023-06-14T12:53:36","modified_gmt":"2023-06-14T10:53:36","slug":"phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction","status":"publish","type":"events","link":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/","title":{"rendered":"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction"},"content":{"rendered":"<section class=\"wp-block-unilux-blocks-free-section section\"><div class=\"container xl:max-w-screen-xl\"><p>You are cordially invited to attend the PhD Defence of\u00a0<strong>Alessandro TEMPERONI<\/strong>\u00a0on\u00a0Monday,\u00a0<strong>the 3rd of July at 10.30 am in Room\u00a01.040, 1st floor\u00a0(MNO) \u2013 Belval Campus.\u00a0<\/strong>You can join the defence also online via\u00a0<a href=\"https:\/\/unilu.webex.com\/unilu\/j.php?MTID=maa2e56b139f71924a3f08721111d7a37\" target=\"_self\" title=\"\" rel=\"noopener\">Webex.<\/a><\/p><p><strong>Members of the defence committee:<\/strong><\/p><ul class=\"ulux-list\"><li class=\"ulux-list-item\">Chair:\u00a0Prof. Dr. Pascal BOUVRY, University of Luxembourg<\/li><li class=\"ulux-list-item\">Vice-chair:\u00a0Prof. Dr. Ulrich SORGER,\u00a0University of Luxembourg<\/li><li class=\"ulux-list-item\">Member (Supervisor):\u00a0Prof. Dr. Martin THEOBALD, University of Luxembourg<\/li><li class=\"ulux-list-item\">Member:\u00a0Prof. Dr. Paolo MERIALDO, Universit\u00e0 Roma Tre<\/li><li class=\"ulux-list-item\">Member:\u00a0Prof. Dr. Gerhard WEIKUM, Max Planck Institute for Informatics<\/li><\/ul><p>\u00a0<\/p><p><strong>Abstract:<\/strong><\/p><p>Since the inception of civilization, the aspiration to create machines capable of thinking has persisted. Over the centuries, this dream has gradually come to fruition, with Artificial Intelligence now emerging as a field with numerous applications and promising research avenues. As a subfield, Deep Learning (DL) is dedicated to developing algorithms that can discern patterns in data, empowering machines to make predictions, draw conclusions, and carry out intricate tasks. This thesis delves into the enhancement of DL performance from various perspectives and vantage points, revealing the complexity of this field and the myriad ways it can be approached, while also highlighting the challenges of navigating through different levels of abstraction and maintaining the focus on the problem at hand.\u00a0<\/p><p>In our exploration, we discuss second-order methods, random embeddings, and relation extraction. We address the initialisation and optimisation of neural networks (NNs) by introducing a new approximated chain rule, which aims to enable rapid and systematic training of NNs. Given that NNs are use-case sensitive, researchers and practitioners must undergo a series of laborious steps before deploying and ultimately releasing a functional model. Although the challenges of training and optimizing NNs are well understood, no single solution exists, and most contemporary approaches rely on simple and empirical heuristics. In part 2, our approximated chain rule for Hessian backpropagation transcends empirical first-order methods and lays a theoretical foundation for optimizing and training NNs. We systematically evaluate our approach through experiments, showcasing the superior efficiency of second-order methods across multiple datasets. In part 3, we shift our focus to analyzing the performance of random embeddings as a crucial tool for dimensionality reduction, with these embeddings playing a significant role in both Machine Learning (ML) and DL algorithms. Our research demonstrates improved bounds for sparse random embeddings compared to previous state-of-the-art techniques, with our bounds exhibiting considerable improvements across a range of real-world datasets. Our analysis strives to bridge the gap between theory and practice, providing robust and provable guarantees for sparse random embeddings while extending to Rademacher random embeddings and offering non-oblivious insights into input data. Lastly, in part 4, we delve into information extraction, specifically targeting relation extraction. By examining the most advanced techniques and tools for extracting and analysing textual data, we demonstrate their applicability in real-world scenarios. Our approach combines distant supervision, few-shot learning, OpenIE, and various language models to enhance the task of relation extraction, showcasing the capabilities of these methods through a simple and efficient approach to extracting relational labels from text. The diverse approaches and strategies discussed in this thesis collectively succeed in augmenting deep learning performance across various scenarios and applications.<\/p><\/div><\/section>","protected":false},"excerpt":{"rendered":"","protected":false},"author":0,"featured_media":1267,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"featured_image_focal_point":[],"show_featured_caption":false,"ulux_newsletter_groups":"","uluxPostTitle":"","uluxPrePostTitle":"","_trash_the_other_posts":false,"_price":"","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"event_start_date":"2023-07-03 10:30:00","event_end_date":"2023-07-03 12:30:00","event_speaker_name":"Alessandro TEMPERONI","event_speaker_link":"","event_is_online":false,"event_location":"Belval Campus, MNO, 1st floor, room 1.040","event_street":"","event_location_link":"","event_zip_code":"","event_city":"","event_country":"LU"},"events-topic":[],"events-type":[],"organisation":[183],"authorship":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v22.3 (Yoast SEO v22.3) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction - SnT - Universit\u00e9 du Luxembourg I Uni.lu<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/\" \/>\n<meta property=\"og:site_name\" content=\"SnT FR\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"2560\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/\",\"url\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/\",\"name\":\"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction - SnT - Universit\u00e9 du Luxembourg I Uni.lu\",\"isPartOf\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg\",\"datePublished\":\"2023-06-14T10:53:36+00:00\",\"dateModified\":\"2023-06-14T10:53:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage\",\"url\":\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg\",\"contentUrl\":\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg\",\"width\":800,\"height\":600},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.uni.lu\/fr\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Interdisciplinary Centre for Security, Reliability and Trust (SnT)\",\"item\":\"https:\/\/www.uni.lu\/snt-fr\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Events\",\"item\":\"https:\/\/www.uni.lu\/snt-fr\/events\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#website\",\"url\":\"https:\/\/www.uni.lu\/snt-fr\/\",\"name\":\"SnT\",\"description\":\"Interdisciplinary Centre for Security, Reliability and Trust I Uni.lu\",\"publisher\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#organization\"},\"alternateName\":\"Interdisciplinary Centre for Security, Reliability and Trust I Universit\u00e9 du Luxembourg\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.uni.lu\/snt-fr\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#organization\",\"name\":\"SnT - Universit\u00e9 du Luxembourg I Uni.lu\",\"alternateName\":\"Interdisciplinary Centre for Security, Reliability and Trust\",\"url\":\"https:\/\/www.uni.lu\/snt-fr\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg\",\"contentUrl\":\"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg\",\"width\":2560,\"height\":2560,\"caption\":\"SnT - Universit\u00e9 du Luxembourg I Uni.lu\"},\"image\":{\"@id\":\"https:\/\/www.uni.lu\/snt-fr\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/school\/snt-lu\/\"],\"description\":\"Interdisciplinary Centre for Security, Reliability and Trust I Uni.lu\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction - SnT - Universit\u00e9 du Luxembourg I Uni.lu","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/","og_locale":"fr_FR","og_type":"article","og_title":"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction","og_url":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/","og_site_name":"SnT FR","og_image":[{"width":2560,"height":2560,"url":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Dur\u00e9e de lecture estim\u00e9e":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/","url":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/","name":"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction - SnT - Universit\u00e9 du Luxembourg I Uni.lu","isPartOf":{"@id":"https:\/\/www.uni.lu\/snt-fr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage"},"image":{"@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage"},"thumbnailUrl":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg","datePublished":"2023-06-14T10:53:36+00:00","dateModified":"2023-06-14T10:53:36+00:00","breadcrumb":{"@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#primaryimage","url":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg","contentUrl":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2023\/06\/phd_defence_enhancing_deep_learning_performance_with_second_order_methods_random_embeddings_and_relation_extraction.jpg","width":800,"height":600},{"@type":"BreadcrumbList","@id":"https:\/\/www.uni.lu\/snt-fr\/events\/phd-defence-enhancing-deep-learning-performance-with-second-order-methods-random-embeddings-and-relation-extraction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.uni.lu\/fr"},{"@type":"ListItem","position":2,"name":"Interdisciplinary Centre for Security, Reliability and Trust (SnT)","item":"https:\/\/www.uni.lu\/snt-fr\/"},{"@type":"ListItem","position":3,"name":"Events","item":"https:\/\/www.uni.lu\/snt-fr\/events\/"},{"@type":"ListItem","position":4,"name":"PhD Defence: Enhancing Deep Learning Performance with Second-order Methods, Random Embeddings, and Relation Extraction"}]},{"@type":"WebSite","@id":"https:\/\/www.uni.lu\/snt-fr\/#website","url":"https:\/\/www.uni.lu\/snt-fr\/","name":"SnT","description":"Interdisciplinary Centre for Security, Reliability and Trust I Uni.lu","publisher":{"@id":"https:\/\/www.uni.lu\/snt-fr\/#organization"},"alternateName":"Interdisciplinary Centre for Security, Reliability and Trust I Universit\u00e9 du Luxembourg","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.uni.lu\/snt-fr\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/www.uni.lu\/snt-fr\/#organization","name":"SnT - Universit\u00e9 du Luxembourg I Uni.lu","alternateName":"Interdisciplinary Centre for Security, Reliability and Trust","url":"https:\/\/www.uni.lu\/snt-fr\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/www.uni.lu\/snt-fr\/#\/schema\/logo\/image\/","url":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg","contentUrl":"https:\/\/www.uni.lu\/wp-content\/uploads\/sites\/13\/2026\/03\/03112647\/SNT_SM-Profile_1600x1600px-scaled.jpg","width":2560,"height":2560,"caption":"SnT - Universit\u00e9 du Luxembourg I Uni.lu"},"image":{"@id":"https:\/\/www.uni.lu\/snt-fr\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/school\/snt-lu\/"],"description":"Interdisciplinary Centre for Security, Reliability and Trust I Uni.lu"}]}},"_links":{"self":[{"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/events\/1266"}],"collection":[{"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/events"}],"about":[{"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/types\/events"}],"replies":[{"embeddable":true,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/comments?post=1266"}],"version-history":[{"count":0,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/events\/1266\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/media\/1267"}],"wp:attachment":[{"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/media?parent=1266"}],"wp:term":[{"taxonomy":"events-topic","embeddable":true,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/events-topic?post=1266"},{"taxonomy":"events-type","embeddable":true,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/events-type?post=1266"},{"taxonomy":"organisation","embeddable":true,"href":"https:\/\/www.uni.lu\/snt-fr\/wp-json\/wp\/v2\/organisation?post=1266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}