PolyU partners with Alibaba to create AI for fashion retailers

The Institute of Textiles and Clothing (ITC) of The Hong Kong Polytechnic University (PolyU) has announced a collaboration with the "vision and beauty team" at Alibaba Group, which specialises in vision intelligence and applications, to establish the "FashionAI Dataset" to improve image-based search and recognition.

A user can take a picture of, and then find fashion items with similar (or even identical) attributes, making product discovery far more powerful.

In a press release, it was explained that current fashion image searching technology used on online platforms is based on the whole fashion image to search the exact or other similar images. However if a customer is interested in some particular fashion attributes of a fashion image and wants to search for other fashion items with these attributes, the current searching technology cannot meet the needs of the customer.

The researchers explain that this limitation is caused by a lack of good data - that is to say that the current datasets aren't constructed utilising professional fashion knowledge and the inability of current technology to train a machine to effectively and accurately understand the fashion attributes of what it's viewing.

A PolyU research team led by professor Calvin Wong, Cheng Yik Hung Professor in Fashion and associate ead of ITC, worked with Alibaba to develop the "FashionAI Dataset" to solve two fundamental problems of the deep learning algorithm: "apparel key points detection" and "attribute recognition".


Key points (e.g. neckline, cuff, waistline) and fashion attributes (e.g. sleeve length, collar type, skirt style) build the foundation for machine learning in understanding fashion images. The establishment of key points and fashion attribute database enables the computer to effectively and efficiently understand the fashion image which is fundamental for deep learning and recognition algorithms.

The accuracy of key points detection is determined by several factors such as the dimension and shape of the apparel, distance and angle of shooting, or even how the apparel is displayed or the model is posing in a photo. These factors can lead to poor key points detection and result in an inaccurate analysis of fashion images by the computer, the researchers said.

"Transforming fashion knowledge into determination of fashion related attributes and fashion item categorisation of the fashion image database is a very complicated and challenging task, while it is the most fundamental task in deep learning applications," said Wong.

Menglei Jia, senior staff engineer at the vision and beauty team at Alibaba commented: "There is a huge potential for AI applications in the fashion industry. In order for AI to understand fashion, which could be very subjective, we need to turn fashion knowledge and experience into language that machine can understand."

Jia added, "We hope to work with academics and the industry alike to explore the wider applications of AI in scenarios including fashion mix-and-match, assisting design and shopping guide, with the aim to bring new values to the fashion industry. The traditional fashion sector should embrace the new retail practice, and we hope FashionAI can be a bridge that connects AI with fashion."