Doctoral researcher Amin Sleimi and senior research scientist Dr Mehrdad Sabetzadeh, part of SnT’s SVV requirements engineering team, won a Best Industrial Paper Award at the 2019 IEEE International Requirements Engineering Conference (RE’19) in South Korea.
We spoke with the researchers as well as their partner, John Dann, from Luxembourg’s Ministère d’Etat – Service central de législation about their work.
Congratulations on your award! It must feel really good to have your peers in the requirements engineering community acknowledge your research through an award like this.
Amin: Thank you! Yes, going to these conferences is always really productive and stimulating. There is so much good feedback you can get through engaging in dialogue at conference workshops. And to have those researchers, the people who understand our work best while still being outsiders, acknowledge that we’ve really accomplished something meaningful is an amazing feeling.
Could you tell us more about the project the paper is based on? What is it that you have accomplished?
Mike: Our project is a tool that, for the first time ever, brings automation to the legal profession.
John: Right. Because up until now, our legal documents had only very simple metadata, and that metadata was fairly superficial, covering only the overarching act but not any of the legal details within it. This meant that users couldn’t easily find what they were looking for and it wasn’t their fault. The problem has been that the machines couldn’t really understand the data either.
Mike: So what our tool does, is it uses NLP [natural language processing] techniques to go through the contents of the catalogue or document database to extract clear, simple, correct answers to intuitively worded questions. That means that instead of getting a list of documents to read when you search “who is considered a taxpayer?” you’d get back a definition that you can work with right away. No manual examination of the documents required. What this does, is it automates part of the legal research work pipeline.
Amin: Exactly. The point here is to develop an automated tool that can take over some of the burdens of legal search, freeing the users to put their energy into more meaningful work. AI [artificial intelligence] is at its best when it works to empower humans to achieve more. And that means we need to design the tool with the human user in mind. The human is an integral part of the design.
Tell me more about that. What does designing a “human-centered” tool entail?
Amin: For one, more is not necessarily more. We needed to simplify and prioritise so that we could meet the end-users where they are, rather than getting carried away with everything that might be technically possible.
Mike: Yes, and the paper that won this award actually argues that there are really just five key question categories that most user queries fall into, and these are what we need to consider when designing this sort of tool.
Amin: Yes. Because the tool really needed to be simple, accessible, and reliable. Everyone needs to be able to use it for it to really work as a democratic tool. And that is a central part of this project.
John: Keeping the end-user in mind through the whole development process has meant that the tool our partnership has produced will be very powerful in the hands of the people who will use it most — legal experts. It has been wonderful to see a new technology grow through such a user-focused process.
What made such a good candidate for this sort of research?legilux.public.lu
John: Well, legal documents already have a pretty good structure to them. And one thing I’ve gotten to see first hand is that artificial intelligence is more about data structure than mathematical magic. Amin and Mike were able to exploit this pre-existing structure to build models that will enable a much more sophisticated search of all the documents in our database. It is a result we could never hope to replicate manually – they are making the impossible possible. It has been an exciting advance for all of us as partners.
What was the most challenging part of your research?
Amin: Just narrowing down the number of questions we used as a launching point, from scores of potential questions to just the five we outline in our question, was a really big step to tackle.
Mike: Yes it was. And then after that, we needed our tool to work very flexibly. To be able to answer such a range of questions, it needs to be able to decide which model to use and when. That was also a really interesting challenge.
John: I was fascinated to learn that most natural language processing machine learning is built specifically for the English language. The challenge of adapting these techniques to French-language documents fascinated me – it was a much bigger task than you might imagine!
Where do you think this research cooperation might lead you next?
John: I want to see users getting the option to send simple questions directly to a chatbot. I would love to see this used as the AI engine behind an interactive tool like that.
Mike: This research is all about helping the users do their work more effectively in the end. Now that the research groundwork has been laid out, user feedback should come to the centreer stage.
Amin: I agree – I’m also looking forward to working with user feedback, and not only as a means to improve what we have already built, but also to provide inspiration for future research directions. Like I said earlier, AI is at its best when it works to empower humans. I am looking forward to developing more AI that does just that.
Find out more about John Dann, Mike Sabetzadeh, and Amin Sleimi.