{"id":10471,"date":"2025-10-30T08:39:28","date_gmt":"2025-10-30T07:39:28","guid":{"rendered":"https:\/\/www.uni.lu\/c2dh-en\/?post_type=events&p=10471"},"modified":"2025-11-05T08:32:00","modified_gmt":"2025-11-05T07:32:00","slug":"hyphe-a-research-oriented-web-crawler","status":"publish","type":"events","link":"https:\/\/www.uni.lu\/c2dh-en\/events\/hyphe-a-research-oriented-web-crawler\/","title":{"rendered":"Hyphe – A research-oriented web crawler"},"content":{"rendered":"\n
\n

\nHands-on History talk with Benjamin Ooghe-Tabanou (m\u00e9dialab – Sciences Po)<\/h2>\n\n\n\n

Developed by Sciences Po m\u00e9dialab as an open source software, Hyphe<\/a> was designed to provide researchers and students with a research oriented crawler to build and enrich corpuses of websites through a qualitative fieldwork methodology. It provides a method and a tool to build a research corpus from web content (web pages and HTTP links) with an innovative approach meant to address two of the main social sciences problems when working with automatized web mining: how to build a theme focused corpus and how to delineate an actor’s presence on the web.<\/p>\n\n\n\n

A step-by-step iterative process supports Hyphe users in dynamically curating and defining \u201cweb entities\u201d in a way that is both granular and flexible by choosing single pages, a subdomain, a combination of websites, etc. The pages residing under these entities are then crawled, in order to extract the outgoing links and part of the textual contents. The most cited \u201cweb entities\u201d can then be prospected manually in order to expand the corpus before visualizing it in the form of a network and exporting it for cleaning and analysis in other tools such as Gephi.<\/p>\n\n\n\n

In partnership with France’s official web archiving teams, Hyphe was recently adapted in order to also crawl web archives from archive.org<\/a> as well the national french library (BnF) and the audiovisual national institute (INA), empowering users to build web corpuses from the past or to complete web corpuses from the live web with archives of disappeared websites<\/p>\n\n\n\n

\nAbout the speaker<\/h3>\n\n\n\n
\n
\n \n\"Benjamin <\/figure><\/div>\n\n\n\n
<\/div>\n\n\n

Trained as a multidisciplinary engineer, Benjamin Ooghe-Tabanou specialises in applying computer science to scientific research. After multiple experiences within the field of astrophysics at Johns Hopkins University in the USA and \u00c9cole Normale Sup\u00e9rieure in France, he enters the social sciences field, first as an Open Data and Parliament transparency activist, cocreating the NGO Regards Citoyens<\/a>, then as a research engineer joining m\u00e9dialab Sciences Po<\/a> in 2012 in order to develop open source tools for social sciences such as the webcrawler Hyphe<\/a>. He supervises m\u00e9dialab’s technical team of research engineers since 2020.<\/p>\n<\/div>\n<\/div>\n\n\n\n