Context-Aware Asset Search for Graphic Design

Context-Aware Asset Search for Graphic Design

Balazs Kovacs1, Peter O'Donovan2, Kavita Bala1, Aaron Hertzmann2
1Cornell University, 2Adobe

IEEE Transactions on Visualization and Computer Graphics

Download: Paper (34.4 MB) , Supplemental Document (1.2 MB), Video (45.2 MB), Supplemental Zip (41.7 MB)

Abstract: Graphic design tools provide powerful controls for expert-level design creation, but the options can often be overwhelming for novices. This paper proposes Context-Aware Asset Search tools that take the current state of the user's design into account, thereby providing search and selections that are compatible with the current design and better fit the user's needs. In particular, we focus on image search and color selection, two tasks that are central to design. We learn a model for compatibility of images and colors within a design, using crowdsourced data. We then use the learned model to rank image search results or color suggestions during design. We found counterintuitive behavior using conventional training with pairwise comparisons for image search, where models with and without compatibility performed similarly. We describe a data collection procedure that alleviates this problem. We show that our method outperforms baseline approaches in quantitative evaluation, and we also evaluate a prototype interactive design tool.


@ARTICLE{8371268, author={B. Kovacs and P. O'Donovan and K. Bala and A. Hertzmann}, journal={IEEE Transactions on Visualization and Computer Graphics}, title={Context-Aware Asset Search for Graphic Design}, year={2018}, volume={}, number={}, pages={1-1}, keywords={ Image color analysis;Search problems;Graphics;Task analysis; Training;Tools;Design tools;Graphic design;machine learning; AB testing;image search;user interfaces;color}, doi={10.1109/TVCG.2018.2842734}, ISSN={1077-2626}, month={}, }