Helping users to find pertinent documents within a big corpus through the use of simple queries on a search engine is a major concern in the information retrieval field. The work presented in this article combines the use of standard natural language processing methods to estimate the relevance of a document to a query with an online preference learning method to infer such kind of pertinence by analyzing the past behavior of other users making similar searches. The first contribution of this article is the proposition of a specific heuristic method, conceived for an open access online juridical corpus, to filter and interpret data collected from the user behavior while navigating on the search engine’s query interface, on the list of results, and on the documents themselves. The second contribution is an original way for combining multiarmed bandit algorithms for learning pertinence from the user implicit feedback with natural language processing techniques in order to define a unique ranking for the search results.