From meaning to perception - exploring the space between word and odor perception embeddings

Imagine you’re smelling a new perfume and want to describe its scent. Most likely you will refer to notes such as lemon, mandarine or musk, that make up the complex scent of a perfume. Our goal in this work is to get vectorial representations (or embeddings²²2In this paper these will be referred to as smell embeddings, note embeddings or odor perception embeddings) for those notes that would capture some relevant properties. We would like these representations to capture that lemon and mandarine are more similar than lemon and musk. In this work we have created such representations using perfume descriptions—each perfume being represented by a list of notes.

But how do we know that the embeddings’ quality is good? The embeddings seem to capture some relevant relationships, as the model finds that e.g. citric components, such as, lemon, mandarine and so on, are relatively similar. Additionally, in order to have a systematic evaluation, we show that the information in our embeddings can be, to some extent, grounded in language, by comparing the relative distances among notes found in our embeddings and in pretrained word embeddings. Furthermore, we expand on this by demonstrating that the measured amount of shared information correlates with how much a word is associated with smell³³3In this paper the terms odor, smell and scent will be used interchangeably. .

In addition, we tried to find the relation between the semantic information in word embeddings and the perceptual information in odor embeddings. We do that by fitting a regressor that maps one embedding space onto another embedding space. Using this regressor, the model is able to suggest a odor perception embedding for any word in the given language.

These notes or perfume components can easily be associated with specific smells. Because of that, we tentatively believe that our model can capture information in the intersection of smell, a sensory modality, and language. This might open up new ways of investigating how our brains form associations between semantic information and sensory experience.

The goal of this work is to create meaningful smell embeddings and to systematically validate them by using a ranking-based approach. Furthermore, we have employed the embeddings in an attempt to understand the aesthetic nature of perfumes, by analyzing the difference between real perfumes and randomly generated ones. We contribute to the scientific community by providing a collection of odor perception embeddings, which are ready to be used by anyone⁴⁴4All our code and embeddings will be made available as a GitHub repository.. We further believe that our findings regarding the composition of perfumes can be extrapolated to other areas where elements are combined in ways that are supposed to provide aesthetic value, such as harmony in music or flavor profiles in cooking.

The article is structured as follows. We start by providing the reader with some background information about perfumes. After that, we discuss the state of the art, in which we introduce works that have tackled the problem of representing perfumes, notes or molecules using different approaches. Then, we present the data and methods that we have employed in our experiments, which are introduced in the next section. We present the results for each of the experiments and discuss those outcomes in the discussion section. As an additional experiment, we present our explorations on mappings between vector spaces within the discussion section. Finally, we close with a conclusion and outline possible future directions.

The scent of a perfume is usually classified in scent family and scent family sub types. Floral, Chypre, Fougère, Marine/Ozonic, Oriental, Citrus, Green, and Gourmand are considered to be the main family types. Examples for family sub types are fruity, spicy, woody, or animalic [9]. In practice, however, there is some variation in what is considered a scent family. [31] point out that grouping odors based on their qualitative similarity bears a huge potential for subjectivity, as basic sensory similarities and categories can be immensely influenced by a person’s background and conception of reality. Furthermore, based on [5], [31] elaborate that, for instance, floral notes might be grouped together not because of a similar smell but rather because of the fact that they all come from flowers. Based on this, such classifications cannot be viewed as a "rendition of truth, but of someone’s rendition of truth" [31]. Aside from its family, a scent can further be described in terms of its notes. A given combination of ingredients is referred to as a composition.

A perfume composition consists of three types of notes: the top notes, the heart notes, the base notes. All of these unfold in different ways over time. The top notes comprise of light, small and highly volatile molecules that evaporate rapidly. The heart notes come to light just before the top notes disappear, which can be anywhere between two minutes up to one hour after a perfume is applied. Base notes emerge when the heart notes disappear. They consist of heavy, large and relatively low-volatile molecules and are often fixatives that support the top and heart notes. They appear only after 30 minutes and can stay up to 24 hours. Given the dynamic nature of a perfume, its perceived smell changes over time [9]⁵⁵5In order to leverage a shared vocabulary with pretrained word embeddings, we made no distinction between e.g. lemon as a top note and lemon as a heart note. The perceived notes of a perfume, however, do not necessarily go back to an ingredient of the same name, e.g. "concrete". Those notes are commonly referred to as fantasy notes. Please find below an example which includes the descriptors regarding the perfume Chance Eau Tendre from Chanel.⁶⁶6https://en.wikipedia.org/wiki/Chanel_Chance

• •

  Top notes: Grapefruit and quince

• •

  Heart notes: Iris absolute and hyacinth

• •

  Base notes: Amber, white musk and cedar wood

The notes assigned to a perfume are semantic descriptors. Although widely applied, there are several questions surrounding these semantic descriptors, e.g. how many descriptors or references are needed in order to appropriately describe a perfume, or what is the right balance of scent families among the descriptors. [31] also recognize the subjectivity involved in this method, since individuals have different olfactory experiences and might use different numbers of descriptors in order to describe an odor. However, they also add that averaging over several subjects provides some reliability. The weak link between smell and language is also found in works [19, 24, 32]. For example, in [19], the authors found that when describing perceptual stimuli across languages, people tend to rely heavily on source-based descriptors. Contrary to the widespread assumption that languages are universally insufficient at describing smell (e.g. [9]), [18] show in a cross-lingual experiment that this might be true for English, but not necessarily for all languages. With regards to English, [17] point out that smell-related language is quite infrequent compared to other sensory modalities such as vision.

Despite this apparent gap, [11] show how the distributional hypothesis and large text corpora can be leveraged to quantify the association between words and odors. Based on similar theoretical assumptions, [8] show that word embeddings trained on general text can be used as features for accurately predicting perceptual ratings of odors.

There have been several attempts to analyze the representation of smell in embedded vectorial spaces. Some of them make use of Natural Language Processing (NLP) resources, others make use of odor characters as perceived by people, and some others combine them all.

In previous work analyzing the odor perception space and the use of language [11], the authors introduce two metrics to describe how strongly a word is associated with olfaction and how specific a word is in its description of odors. Furthermore, they introduce a two-dimensional space that characterizes perceptual olfactory connotations of English-language odor descriptors. We opted for a similar approach but decided to use a different evaluation procedure, which we describe in section 5. The work by Hörberg et al. [10] is also rather related to ours. They propose a data-driven approach that automatically identifies odor descriptors in English, and then derive their semantic organization on the basis of their distributions in natural texts. Other authors [33] analyze perfume notes (making a distinction between top, heart and base notes) and extract the Principal Components. They propose a visualization which is a modified version of the Hexagon of Fragrance Families, a sensory map that visualizes perfumes in a polygon. Each side of the polygon represents odor characteristics.

There are a number of works that make an attempt to predict odor descriptors based on molecules. A number of these works were presented as part of the DREAM olfaction prediction challenge [13] or make use of the publicly available data set [14]. The authors in [28, 27] trained models to predict the odor of molecules. The problem, called quantitative structure-odor relationship (QSOR) modeling, is tackled using graph neural networks. Khan et al. [15] built an olfactory perceptual space and a molecular physicochemical space. They predict pleasantness of different molecules based on a Principal Component Analysis (PCA) reduction of the character space defined by Dravnieks [6]. Other researchers [23, 8] made use of vectorial representations of words for molecule-based odor prediction. While [23] used Word2Vec embeddings [21], [8] made use of FastText word embeddings [4].

As suggested by researchers in [7], the website Basenotes⁷⁷7www.basenotes.net provides data for thousands of commercially available perfumes. For this study, 42773 perfumes were obtained from the website. The top notes, heart notes and base notes of each perfume were collected⁸⁸8We do not have any explicit information about the number of authors involved. However, we have no reason to assume it was only one.. We excluded perfumes with less than 3 listed notes or missing note category labels, which reduced the number of perfumes to 26253. After extraction the lists of notes were lower-cased and all punctuation was removed (e.g. in order to match Ylang-ylang and ylang-ylang). We applied no further normalization processes. Consequently, notes such as sicilian lemon and lemon were treated as distinct notes. The final dataset consisted of 12550 notes distributed over 26253 perfumes with average number of notes $\mu=8.6$ ($\sigma=4.1$). Table 1 shows the 10 most frequent notes.

This section covers the two main experiments of our study.

In this section, we discuss the performed experiments and their results followed by an additional tentative experiment, where we explore the possibility of mappings from the word embedding space to the smell embedding space. In experiment 1a, in general, we observed that the odor perception embeddings contain some information about smell. We evaluated this by calculating the Rank Biased Overlap (RBO) for each word in the shared vocabulary, effectively measuring the similarity between two rankings. One ranking was the one with the closest words from the smell embeddings (perfume) and the other ranking came from word embeddings (language). When comparing these RBOs with the ones obtained between smell embeddings and randomly shuffled word embeddings, we observed that there is a small but significant difference between the RBOs. This suggests, then, that there is an amount of shared information between the embedding spaces. However, the shared information seems to be rather limited which can be explained by the fact that general English does not express a large amount of olfactory information or the polysemy of the word embeddings (e.g. "orange" can be both a color and a smell, which probably increases the difference between "lemon" and "orange" in word embedding space).

Furthermore, these embeddings do not only seem to encode intuitive and established information, e.g. that lavender and lavandin are similar, but they also seem to capture subtle information, or information that may not be available at a first glance. For example, we found that musk and jasmine are relatively similar. While there are sources highlighting the "muskyness" of jasmine, this characteristic is not reflected in traditional odor classification schemes. However, when looking at the ranking based on closest words, we noticed a trend: The most similar notes also seem to be the most frequent ones in a number of cases. We still need to analyze the effect of frequency in the embeddings, and we may need to account for that effect. A weighting scheme, similar to TF-IDF¹⁸¹⁸18Term-Frequency - Inverse Document Frequency. could be applied as a possibility.

Concerning experiment 1b, the positive correlation between the RBOs and olfactory association indicates a tendency that the agreement between the spatial relationships between word embeddings and smell embeddings grows with the amount of olfactory information in a word embedding. This seems to support our assumption that at least a part of the little agreement can be explained by the general lack of olfactory information in English.

The goal of Experiment 2 was to get a first insight in whether the smell embeddings can potentially be used in order to model perfume aesthetics. Our results show a tendency that the notes of real perfumes are more evenly distributed around their centroid compared to randomly sampled perfumes. Although significant, the difference of the two groups was rather small, indicating that our measure only explains a small fraction of what makes perfumes pleasant. In general, these results encourage further research and analysis.

In this work, we used Word2vec as a method for creating meaningful perfume note representations. We tested the quality of those embeddings by following a rank based metric (Rank Biased Overlap). Besides, experiment 1b suggests that the words with a higher RBO, or higher agreement in the ranking between note and word embedding spaces, are somehow more related to olfaction. Looking at the embeddings of particular notes, we discovered that they contain meaningful similarities to other notes that seem to capture both salient and more subtle similarities. Since these similarities sometimes transcend traditional categories, we believe that the embeddings presented in this paper could potentially contribute to new ways of odor categorization. Furthermore, our findings suggest that, in relation to distributional semantics, aesthetics in data with complex structures might be an equivalent concept to semantics in language.

Additionally, we have performed an initial experiment in order to evaluate the potential use of our embeddings for modelling the pleasantness of a perfume. Our results suggest a tendency that the notes of real perfumes are more evenly distributed around their centroid compared to randomly sampled perfumes, but we believe this needs further analysis as the difference is relatively small. Since our results were nonetheless significant, we cautiously read them as encouragement for further exploration of geometric properties of perfumes in our embedding space in relation to their pleasantness.

To conclude, we explored the use of regression models for creating linear mappings between embedding spaces. This way, given a word in the English language, we can propose a perfume note based on this mapping. However, this approach needs further improvement.

In this work we did not associate each perfume with a specific gender, and we therefore considered all the perfumes’ population as a whole. It would be very interesting to analyze the perfumes considering their gender association. Furthermore, it would also be relevant to investigate the grammatical gender of the odor descriptors, as done previously by Speed and others [29]. In order to do this, however, we would have to analyze the odor descriptors and their embeddings in languages other than English, in which words have an associated gender, such as German, Spanish, French, and so on.

With regards to preprocessing, there is considerable room for improvement: Applying lemmatization to all words might provide to be useful. Additionally, we are planning on investigating the importance of the choice of hyperparameters of the word2vec algorithm in this setting, most notably the model architecture (CBOW vs. skip gram) and the size of the context window.

Furthermore, another possibility to improve our representations could be to include molecule information to these smell embeddings. In order to do so, we need a mapping between notes and molecules.

With regards to the linear mapping experiments, we believe that using more advanced mapping algorithms could make our results better and also that we would be able to provide more accurate odors for other non-odor related words. A possibility could be to follow the work on Unsupervised Machine Translation [1, 16] and also Bilingual Lexical Induction [2]. In relation to this, we would like to have a proper and robust way for evaluating these mappings, besides qualitative and intuitive evaluations.

Last but not least, we want to continue trying to match different perception and language spaces. We have analyzed and mapped the spaces related to language and olfaction. Our intention is to do similar experiments with other sensory modalities, such as vision, taste or hearing.