ICDE 2018: Recommendations for All : Solving Thousands of Recommendation Problems Daily

I'm excited to share that our paper on scaling a recommendation servicewas accepted to ICDE 2018! An ML+Systems perspective.
Recommender systems are a key technology for many online services includinge-commerce, movies, music, and news. Online retailers use product recommender systemsto help users discover items that they may like. However, building a large-scaleproduct recommender system is a challenging task. The problems of sparsity andcold-start are much more pronounced in this domain. Large online retailers have usedgood recommendations to drive user engagement and improve revenue, but the complexityinvolved is a roadblock to widespread adoption by smaller retailers.
In this paper, we tackle the problem of generating product recommendations for
tens of thousands of online retailers. Sigmund is an industrial-scale system forproviding recommendations as a service. Sigmund was deployed to production in early2014 and has been serving retailers every day. We describe the design choices that wemade in order to train accurate matrix factorization models at minimal cost.We also share the lessons we learned from this experience -- both from a machinelearning perspective and a systems perspective. We hope that these lessons areuseful for building future machine-learning services.