Expanding our earlier work on choice overload in related-article recommendations, we analyzed a digital library to figure out the ideal number of recommendations to show to users. Some of our metrics point toward 5-6 items as the ideal number of recommendations to display. You can find the full paper here (PDF).
Choice Overload describes a situation in which a person has difficulty in making decisions due to too many options. We examine choice overload when displaying related-article recommendations in digital libraries, and examine the effectiveness of recommendations algorithms in this domain. We first analyzed existing digital libraries, and found that only 30% of digital libraries show related-article recommendations to their users. Of these libraries, the majority (74%) displays 3-5 related articles; 28% of them display 6-10 related articles; and no digital library displayed more than ten
related-article recommendations. We then conducted our experimental evaluation through GESIS’ digital library Sowiport, with recommendations delivered by recommendations-as-a-service provider Mr. DLib. We use four metrics to analyze 41.3 million delivered recommendations: click-through rate (CTR), percentage of clicked recommendation sets (clicked set rate, CSR), average clicks per clicked recommendation set (ACCS), and time to rst click (TTFC), which is the time between delivery of a set of recommendations to the first click. These metrics help us to analyze choice overload
and can yield evidence for finding the ideal number of recommendations to display. We found that with increasing recommendation set size, i.e., the numbers of displayed recommendations, CTR decreases from 0.41% for one recommendation to 0.09% for 15 recommendations. Most recommendation sets only receive one click. ACCS increases with set size, but increases more slowly for six recommendations and more. When displaying 15 recommendations, the average clicks per set is at a maximum (1.15). Similarly, TTFC increases with larger recommendation set size, but increases more slowly for sets of more than five recommendations. While CTR and CSR do not indicate choice overload, ACCS and TTFC point towards 5-6 recommendations as being optimal for Sowiport. Content-based filtering yields the highest CTR with 0.118%, while stereotype recommendations yield the highest ACCS (1.28). Stereotype recommendations also yield the highest TTFC. This means that users take more time before clicking stereotype recommendations when compared to recommendations based on other algorithms.
