Abstract

A good visualisation method can greatly enhance human-machine collaboration in target contexts. To aid the optimal selection of visualisations for users, visualisation recommender systems have been developed to provide the right visualisation method to the right person given specific contexts. A visualisation recommender system often relies on a user study to collect data and conduct analysis to provide personalised recommendations. However, a user study without employing an effective experimental design is typically expensive in terms of time and cost. In this work, we propose a prediction-oriented optimal design to determine the user-task allocation in the user study for the recommendation of visualisation methods. The proposed optimal design will not only encourage the learning of the similarity embedded in the recommendation responses (i.e., users' preference), but also improve the modelling accuracy of the similarities captured by the covariates of contexts (i.e., task attributes). A simulation study and a real-data case study are used to evaluate the proposed optimal design.

References

1. Anava, O., Golan, S., Golbandi, N., Karnin, Z., Lempel, R., Rokhlenko, O., & Somekh, O. (2015). Budget-constrained item cold-start handling in collaborative filtering recommenders via optimal design. In Proceedings of the 24th international conference on world wide web (pp. 45–54). Florence, Italy: International World Wide Web Conferences Steering Committee. [Google Scholar]
2. Bednarik, R., Vrzakova, H., & Hradis, M. (2012). What do you want to do next: A novel approach for intent prediction in gaze-based interaction. In Proceedings of the symposium on eye tracking research and applications (pp. 83–90). Association for Computing Machinery. [Crossref], [Google Scholar]
3. Bhat, N., Farias, V. F., Moallemi, C. C., & Sinha, D. (2020). Near-optimal AB testing. Management Science, 66(10), 4477–4495. https://doi.org/10.1287/mnsc.2019.3424 [Crossref], [Web of Science ®], [Google Scholar]
4. Bogdanov, D., Haro, M., Fuhrmann, F., Xambó, A., Gómez, E., & Herrera, P. (2013). Semantic audio content-based music recommendation and visualization based on user preference examples. Information Processing & Management, 49(1), 13–33. https://doi.org/10.1016/j.ipm.2012.06.004 [Crossref], [Web of Science ®], [Google Scholar]
5. Box, G. E., & Draper, N. R. (1959). A basis for the selection of a response surface design. Journal of the American Statistical Association, 54(287), 622–654. https://doi.org/10.1080/01621459.1959.10501525 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
6. Boyd, S., Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press. [Crossref], [Google Scholar]
7. Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717. https://doi.org/10.1007/s10208-009-9045-5 [Crossref], [Web of Science ®], [Google Scholar]
8. Chakraborty, S., Zhou, J., Balasubramanian, V., Panchanathan, S., Davidson, I., & Ye, J. (2013). Active matrix completion. In 2013 IEEE 13th international conference on data mining (pp. 81–90). IEEE. [Crossref], [Google Scholar]
9. Chen, X., & Jin, R. (2017). Statistical modeling for visualization evaluation through data fusion. Applied Ergonomics, 65, 551–561. https://doi.org/10.1016/j.apergo.2016.12.016 [Crossref], [Web of Science ®], [Google Scholar]
10. Chen, X., & Jin, R. (2018). Data fusion pipelines for autonomous smart manufacturing. In 2018 IEEE 14th international conference on automation science and engineering (CASE) (pp. 1203–1208). IEEE. [Crossref], [Google Scholar]
11. Chen, X., & Jin, R. (2020). AdaPipe: A recommender system for adaptive computation pipelines in cyber-manufacturing computation services. IEEE Transactions on Industrial Informatics, 1–1. https://doi.org/10.1109/TII.2020.3035524 [Google Scholar]
12. Chen, X., Lau, N., & Jin, R. (2020). PRIME: A personalized recommendation for information visualization methods via extended matrix completion. ACM Transactions on Interactive Intelligent Systems, 11(1), 1–30. https://doi.org/10.1145/3366484 [Google Scholar]
13. Chen, X., Sun, H., & Jin, R. (2016). Variation analysis and visualization of manufacturing processes via augmented reality. In Presented at the IISE annual conference. Proceedings. IISE. [Crossref], [Google Scholar]
14. Crary, S. B., Hoo, L., & Tennenhouse, M. (1992). I-optimality algorithm and implementation. In Computational statistics (pp. 209–214). Springer. [Crossref], [Google Scholar]
15. Deng, X., & Jin, R. (2015). QQ models: Joint modeling for quantitative and qualitative quality responses in manufacturing systems. Technometrics, 57(3), 320–331. https://doi.org/10.1080/00401706.2015.1029079 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
16. Deng, X., & Tsui, K. W. (2013). Penalized covariance matrix estimation using a matrix-logarithm transformation. Journal of Computational and Graphical Statistics, 22(2), 494–512. https://doi.org/10.1080/10618600.2012.715556 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
17. Donoho, D. L., Elad, M., & Temlyakov, V. N. (2005). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory, 52(1), 6–18. https://doi.org/10.1109/TIT.2005.860430 [Crossref], [Web of Science ®], [Google Scholar]
18. Jin, R., & Deng, X. (2015). Ensemble modeling for data fusion in manufacturing process scale-up. IIE Transactions, 47(3), 203–214. https://doi.org/10.1080/0740817X.2014.916580 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
19. Jones, B., Silvestrini, R. T., Montgomery, D. C., & Steinberg, D. M. (2015). Bridge designs for modeling systems with low noise. Technometrics, 57(2), 155–163. https://doi.org/10.1080/00401706.2014.923788 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
20. Joseph, V. R. (2016). Space-filling designs for computer experiments: A review. Quality Engineering, 28(1), 28–35. https://doi.org/10.1080/08982112.2015.1100447 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
21. Kang, X., & Deng, X. (2020). On variable ordination of Cholesky-based estimation for a sparse covariance matrix. Canadian Journal of Statistics. https://doi.org/10.1002/cjs.11564 [Crossref], [Web of Science ®], [Google Scholar]
22. Kiefer, J., & Wolfowitz, J. (1959). Optimum designs in regression problems. The Annals of Mathematical Statistics, 30(2), 271–294. https://doi.org/10.1214/aoms/1177706252 [Crossref], [Google Scholar]
23. Kunkel, J., Loepp, B., & Ziegler, J. (2017). A 3D item space visualization for presenting and manipulating user preferences in collaborative filtering. In Proceedings of the 22nd international conference on intelligent user interfaces (pp. 3–15). Association for Computing Machinery. [Crossref], [Google Scholar]
24. Laake, P. (1975). On the optimal allocation of observations in experiments with mixtures. Scandinavian Journal of Statistics, 2, 153–157. https://www.jstor.org/stable/4615596 [Google Scholar]
25. Mak, S., & Xie, Y. (2017). Active matrix completion with uncertainty quantification. Preprint. arXiv:1706.08037. [Google Scholar]
26. Melville, P., & Sindhwani, V. (2010). Recommender systems. Encyclopedia of Machine Learning, 1, 829–838. [Google Scholar]
27. Mohri, M., & Talwalkar, A. (2011). Can matrix coherence be efficiently and accurately estimated? In Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 534–542). JMLR Workshop and Conference Proceedings. [Google Scholar]
28. Mutlu, B., Veas, E., & Trattner, C. (2016). Vizrec: Recommending personalized visualizations. ACM Transactions on Interactive Intelligent Systems, 6(4), 1–39. https://doi.org/10.1145/2983923 [Crossref], [Web of Science ®], [Google Scholar]
29. Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7(4), 308–313. https://doi.org/10.1093/comjnl/7.4.308 [Crossref], [Web of Science ®], [Google Scholar]
30. Pukelsheim, F. (2006). Optimal design of experiments. SIAM. [Crossref], [Google Scholar]
31. Ricci, F., Rokach, L., & Shapira, B. (2015). Recommender systems: Introduction and challenges. In Recommender systems handbook (pp. 1–34). Springer. [Crossref], [Google Scholar]
32. Ruchansky, N., Crovella, M., & Terzi, E. (2015). Matrix completion with queries. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1025–1034). Association for Computing Machinery. [Crossref], [Google Scholar]
33. Sagnol, G., & Harman, R. (2015). Computing exact d-optimal designs by mixed integer second-order cone programming. Annals of Statistics, 43(5), 2198–2224. https://doi.org/10.1214/15-AOS1339 [Crossref], [Web of Science ®], [Google Scholar]
34. Santner, T. J., Williams, B. J., Notz, W. I., & Williams, B. J. (2003). The design and analysis of computer experiments (Vol. Springer. [Crossref], [Google Scholar]
35. Schein, A. I., Popescul, A., Ungar, L. H., & Pennock, D. M. (2002). Methods and metrics for cold-start recommendations. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval (pp. 253–260). Association for Computing Machinery. [Crossref], [Google Scholar]
36. Sharma, G. (2017). Pros and cons of different sampling techniques. International Journal of Applied Research, 3(7), 749–752. [Google Scholar]
37. Spence, R. (2001). Information visualization (Vol. Springer. [Google Scholar]
38. Vartak, M., Huang, S., Siddiqui, T., Madden, S., & Parameswaran, A. (2017). Towards visualization recommendation systems. ACM SIGMOD Record, 45(4), 34–39. https://doi.org/10.1145/3092931.3092937 [Crossref], [Web of Science ®], [Google Scholar]
39. Voigt, M., Pietschmann, S., Grammel, L., & Meißner, K. (2012). Context-aware recommendation of visualization components. In The fourth international conference on information, process, and knowledge management (EKNOW) (pp. 101–109).  IARIA XPS Press. [Google Scholar]
40. Vozniuk, A., Rodriguez Triana, M. J., Holzer, A. C., & Gillet, D. (2016). Combining content analytics and activity tracking to identify user interests and enable knowledge discovery. In 6th international workshop on personalization approaches in learning environments (PALE). ACM. [Crossref], [Google Scholar]
41. Xu, Z. (2018). The minimal measurement number for low-rank matrix recovery. Applied and Computational Harmonic Analysis, 44(2), 497–508. https://doi.org/10.1016/j.acha.2017.01.005 [Crossref], [Web of Science ®], [Google Scholar]
42. Zhou, H., Li, L., & Zhu, H. (2013). Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association, 108(502), 540–552. https://doi.org/10.1080/01621459.2013.776499 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]