Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social Connections
Abstract.
Ubiquitous computing encapsulates the idea for technology to be interwoven into the fabric of everyday life. As computing blends into everyday physical artifacts, powerful opportunities open up for social connection. Prior connected media objects span a broad spectrum of design combinations. Such diversity suggests that people have varying needs and preferences for staying connected to one another. However, since these designs have largely been studied in isolation, we do not have a holistic understanding around how people would configure and behave within a ubiquitous social ecosystem of physically-grounded artifacts. In this paper, we create a technology probe called Social Wormholes, that lets people configure their own home ecosystem of connected artifacts. Through a field study with 24 participants, we report on patterns of behaviors that emerged naturally in the context of their daily lives and shine a light on how ubiquitous computing could be leveraged for social computing.
1. Introduction
Family and friends often desire to stay connected with each other over distance. Currently, our way of staying connected with others is device-centric, using smartphones, tablets, and computers. With the onset of smart homes, smart materials, and smart cities, however, the field of computing is slowly marching toward the vision of ubiquitous computing (Weiser, 1991), in which technology will evolve to become much more interwoven into our surrounding environment. A major opportunity for ubiquitous computing is to promote staying connected with others in an environment-centered way, repurposing the surrounding environment to serve as connection points with family and friends. Very little work, however, has explored how ubiquitous computing and a potentially large ecosystem of connected endpoints might foster this sort of social connection.
A significant stream of work exploring social connection endpoints is that of Tangible Bits (Ishii and Ullmer, 1997), Ambient Media, Ambient Telepresence, and the idea that having connected objects or virtually connected spaces can be a powerful means of feeling close or even together with family and friends over distance. In these works, presence and activity information are embodied in physical artifacts including objects, surfaces, and spaces. For example, in AmbientROOM (Ishii et al., 1998), information about a loved one’s activities is relayed via the movement of a computer mouse and the projection of water ripples on the ceiling. Since then, numerous other systems have emerged that have been shown to be effective in supporting a sense of presence, awareness, and social connection in work and domestic life. These have taken on multiple forms, ranging from small household objects like candles, picture frames and desktop toys (Chang et al., 2001; Brewer et al., 2005; Kim et al., 2015; Häkkilä et al., 2018) to augmented furniture (Dey and de Guzman, 2006; Siio et al., 2002), and have evolved to relay various types of information, including mood, presence and activity between people (Hassenzahl et al., 2012).
The vast array of design combinations is indicative that people may have a broad spectrum of needs and preferences for how to connect with one another via more ubiquitous social connections. Prior work echoes this sentiment, suggesting that “one-size-fits-all solutions do not work” for sharing aspects of domestic life (Neustaedter et al., 2015) and that people develop unique meanings when using augmented objects (Ambe et al., 2017; Hassenzahl et al., 2012). What we notice, however, is that existing social connection systems are often designed for use in a fixed configuration, with few options for customization once they have been deployed. Many, for example, are based on a fixed pair of augmented objects with a dedicated set of specialized multi-modal capabilities. As such, user studies with these systems can reveal the impact those particular designs on social connection, but cannot lead to insights into user preferences beyond the scope of their specific design configurations. Additionally, while technology is maturing to a state where it might be possible to create a landscape of multiple connected artifacts, little is known about how users would perceive and use a broader ecosystem of distributed and physically grounded artifacts for social connection.
Therefore, in this paper, we take a step back to explore what people’s behaviors and resulting preferences would be if given the power to create their own ecosystems of connected objects for social connection. To do so, we create a customizable and scalable technology probe (Hutchinson et al., 2003) called Social Wormholes and deploy it in a field study with 12 pairs of friends (24 participants) over two weeks. Figure 1 illustrates Social Wormholes. To make Social Wormholes customizable and extensible for our investigation, we base its implementation on augmented reality (AR). With the platform, each person can make various physical artifacts in their homes points of social connection to their respective friend, by attaching printed AR markers. The markers are analogous to the concept of Tabs in Mark Weiser’s original envisionment of ubiquitous computing (Weiser, 1991), turning any object into a social connection point quickly and cheaply. With the use of AR glasses and dedicated apps, connections can be established between the markers, and the artifacts can serve as mediums through which the friends send and receive snippets of information to each other.
The flexibility of the probe makes it such that a pair of friends may (1) customize which objects in their homes become connected objects, (2) determine how these artifacts are mapped to one another, and (3) elect how many concurrent connections they use with their remote friend in the given period of time. Using this approach, we allow patterns of behavior to organically emerge, and capture this through a system of surveys, system logs and interviews. Based on an analysis, we report a spectrum of user preferences towards social connection. Furthermore, we outline a set of design recommendations for system features that may help to best serve these different populations. In summary, we contribute:
-
(1)
The design and implementation of an AR-based technology probe of a distributed and physically-grounded social connection system for use between a pair of remote friends;
-
(2)
Insights from a field study with 12 pairs of friends (24 participants) on what types of physical artifacts people favor for social connection, how they could be mapped to one another, their perceptions around having a flexible number of connected artifacts, and different scenarios that can be supported with this infrastructure;
-
(3)
An outline of different patterns of behavior users exhibit towards physically-grounded social connections, and associated design implications.
2. Related Work
There is a rich history of work on creating interconnected physical or digital objects for social connection, stemming from the visionary concept of Ubiquitous Computing (Weiser, 1991; Weiser and Brown, 1997). We provide an overview of prior works in this space that leverage physical artifacts for social connection, and also discuss frameworks for understanding how people design and interact with such technologies.
2.1. Ubiquitous Physical Artifacts for Social Connection
Many prior works focus on improving social connection for people in Long Distance Relationships (LDR) or more generally, people who live in separate households. Examples of such systems include household fixtures and furniture such as bathroom mirrors, bed-side drawers, stools (Schmeer and Baffi, 2010; Siio et al., 2002; Dey and de Guzman, 2006), and household objects such as stuffed toys, candles, tools, picture frames and desktop toys (Fong et al., 2013; Häkkilä et al., 2018; Davis-Owusu et al., 2019; Chang et al., 2001; Kim et al., 2015). Some works created entirely new physical artifacts such as novel desktop toys and radios (Brewer et al., 2005; Heshmat et al., 2020) rather than augmenting existing ones. As pointed out by Li et al. (Li et al., 2018), a majority of systems rely on a single type of device for bidirectional communication, meaning they used a symmetric pairing of objects. There are also a few systems that feature non-matching connected objects. Examples of asymmetric systems, include Shared Wind (Yu and Inakage, 2022), a uni-directional communication system with sender and receiver curtains, and Flex-N-Feel (Singhal et al., 2017), which comprised a flex-sensing sender glove and a vibrotactile receiver glove. A pioneering work, AmbientROOM, mapped (Ishii et al., 1998) a pet’s movement is mapped to a visual projection of ripples on the ceiling.
User studies for these systems demonstrated that they were helpful in fostering improved feelings of presence and social connection. However, there is still much more room to investigate how people would use and be impacted by having ubiquitous socially-connected physical artifacts dispersed throughout one’s environment. Therefore, with our technology probe, we enable users to designate existing artifacts in their homes as connection endpoints, establish how they are mapped to their remote friends’ artifacts, and manage how many concurrent connections they use within their personalized ecosystem of connected artifacts. In this way, we can begin to gather insights into how people behave given the ability to create a ubiquitous constellation of social connections.
2.2. Understanding Technology-Mediated Social Connection
Many previous researchers have explored the design space for technology-mediated social connection. As Hassenzahl et al. (Hassenzahl et al., 2012) explains, the feeling of “relatedness” is an integral psychological human need that technologies can support by supporting people in awareness, expressivity, physicalness, gift giving, joint action and memories.
Given the wealth of strategies, many efforts have been made not only to create instances of these technological systems, but to understand and map out their design space. For instance, many researchers (Matthews et al., 2003; Pousman and Stasko, 2006; Gooch and Watts, 2011) have identified important design dimensions for ambient-media systems, ranging from information capacity and notification levels, to sensory mediums, personalization and more. In a systematic review of 150 articles of unconventional user interfaces for LDR emotional communication (i.e. excluding mobile apps) by Li et al. (Li et al., 2018), it was found that non-symmetric pairings of devices (meaning that the two objects are not of the same kind) and longer-duration studies of technologies for social connection in real-life use contexts remain underexplored and under-represented. Since we wish to expand knowledge in how a ubiquitous computing approach can be used for social connection, it was particularly necessary for us to incorporate these two aspects as part of our investigation.
We base our investigative approach on a technology probe (Hutchinson et al., 2003), a research technique well-suited to studying social connection due to its ability to reveal surprising insights. For instance, Lottridge et al. (Lottridge et al., 2009) uncovered the power of promoting sharing during “empty moments” to nurture long-distance relationships. In another example, Judge et al. (Judge et al., 2010, 2011) explore video-based platforms for social connection between remote families. They discovered that Family Window (Judge et al., 2010), which connected two households, triggered routine sharing of everyday moments, but that Family Portals (Judge et al., 2011), which connected three households, did not trigger such routine sharing. Grivas (Grivas, 2006) experimented with establishing an imaginary “merge” of two homes using arrangements of physical LED prototypes in people’s homes, and found that incorporating people’s spatial knowledge of each other’s places could evoke intimacy and a sense of presence. Unlike these previous technology probes, Social Wormholes gives users a high degree of control over how to connect their space with their friend’s space (users can connect anything with anything), allowing us to understand how people feel and behave with the ability to configure a ubiquitous computing environment specifically for social connection. Our insights can inform the design of ubiquitous systems for social connection.
Other systems have explored AR as a means of sending and leaving physically-grounded messages for friends and colleagues. Specifically, they explore the paradigm of “leaving behind” AR text (Lin et al., 2011; Nassani et al., 2015; Bâce et al., 2016) and audio messages (Langlotz et al., 2013) at places so that others see them there later. Unlike this body of work, our technology probe is centered around an ecosystem of connected physical artifacts rather than virtual AR objects being left behind. Other recent works in AR have explored using AR glasses for communication. For example, ARwand (Lee et al., 2022) enabled people to compose and send virtual content to be rendered on a remote friend’s AR glasses, and for which they can see the friend’s reactions. ARcall (Surale et al., 2022) was a platform for a remote friend to “drop in” to see what a remote friend sees, and to inject an AR asset into their view to be seen via AR glasses. In contrast to these works, our AR-glasses-based technology probe does not involve direct sending of content to a person wearing the glasses, but rather allows people to transmit content anchored to virtually connected physical objects. The probe employs the use of AR to make the marginal cost of establishing new connected artifacts small—just more AR markers (pieces of paper)—giving our participants a chance to live in a large ecosystem of connected objects.
3. Technology Probe: Social Wormholes
Social Wormholes is an AR-glasses based system that serves as a flexible platform for social connection between two people. It has three components as seen in Figure 2: (1) a set of printed markers (i.e. wormhole images) that users can attach to objects or locations of their choice and distribute within their physical space; (2) a setup app for users to establish new wormhole connections or manage existing connections; and (3) the main AR-glasses application for users to send and receive Sparkles and Ghosts.
We arrived at Social Wormholes through an iterative design process. Our research team considered many radically different designs for the technology probe, undergoing several rounds of brainstorming and creating several low-fidelity prototypes to develop our ideas.
At the start of the design process, we outlined three base requirements that the technology probe should fulfill in order for us to explore how a potentially large ecosystem of connected artifacts can support social connections. First, the technology probe should make it possible to convert any existing object or place into a medium for social connection. Second, the probe should scale to accommodate multiple artifacts while remaining cost-effective to deploy. Last, the probe should allow for communication “on-the-go”, wherein a user can use the system impromptu in a way that does not impede their current activity. Users should be able to interact directly with connected artifacts and stumble upon transmitted messages naturally in their environment, rather than intentionally check a separate device.
During the iterative process, we considered designs such as attachable hardware modules, a large network of touchscreen tablets that could be placed next to objects, and a smartphone app that utilizes AR to overlay virtual content onto objects. We converged on a system that combines AR glasses with paper markers because it satisfied our requirements best. The paper markers allow users to convert any object into a medium for social connection (akin to Mark Weiser’s tabs (Weiser, 1991)) and can be scaled inexpensively to make more connected artifacts, while the AR glasses could be used to engage with the system while also carrying out other activities.
3.1. Design and Implementation
We printed wormhole marker images at a size of 88 inches on standard letter-sized sheets of paper. We tested the markers under various lighting conditions and viewing angles, and shipped five markers to each participant in order to ensure marker tracking accuracy and robustness. We developed two apps for our technology probe. First, we built a phone-based setup app (see Figure 4). Users can establish up to five one-to-one connections between wormholes of their choice. They can remove and change wormholes connections and setups by editing the connections in the app. In addition, we built a connection app on AR glasses (see Figure 3). With the connection app, users can transmit and receive content between the one-to-one wormholes connections they created. While other connection types may be valuable (i.e. one-to-many, many-to-many), we used this as a starting point to keep things simple for users (it is a common mapping pattern for prior systems, as noted in Section 2). We developed both applications as Snap lenses using Lens Studio111https://lensstudio.snapchat.com/. The Social Wormholes system can operate on an iPhone 8 or later and on Snap Next Generation Spectacles. As part of the study, we lent AR glasses to the participants and confirmed that they would have access to a compatible smartphone. The setup app connects to the AR glasses via Bluetooth.
Our system enables two formats of transmissions, Sparkles and Ghosts, as can be seen in Figure 3. Sparkles are meant to be a lightweight form of communication. A user transmits Sparkles to a friend by simply looking at a wormhole while wearing the AR glasses. Once the gaze is registered, Sparkles are transmitted to the friend’s corresponding wormhole. The next time the remote friend passes by their corresponding wormhole endpoint, they can receive the Sparkles by simply gazing at the wormhole marker while wearing their AR glasses. Once their gaze is recognized by their AR glasses, it plays an animation that shows the Sparkles in the form of particle bursts hovering above the wormhole opening on their end. Sparkles can act as an indicator that their remote friend was near their corresponding wormhole endpoint of a particular wormhole connection. Ghosts are a higher-fidelity form of communication that sends not just a single bit of information (as Sparkles do) but some richer context about the user’s surrounding environment as well. A Ghost consists of the image of an object in the user’s field of view accompanied by a short five-second audio recording. To begin capturing a Ghost, the user must look at a wormhole and perform a single-finger swipe-forward gesture on the touch pad of the AR glasses. This starts a five-second countdown, during which the surrounding audio is recorded (e.g., ambient noises, speech). At the end of the five seconds, an object in the user’s field of view is captured using the forward-facing camera of the AR glasses. The captured object along with the audio recording is then sent to the friend’s corresponding wormhole endpoint. The Ghost (captured object + audio recording) will appear for the friend the next time they look at their corresponding wormhole. This sequence is pictured in Figure 3.
In order to facilitate the exchange of information between two connected users in a pair, we also implemented two cloud databases on Amazon Web Services (AWS) to store and readout Sparkles and Ghosts contents that users create. Specifically, we used DynamoDB to store wormhole endpoints connections, users, Sparkles and Ghosts transmission data, and S3 to store content like Ghosts images and audio recording. Both our phone app and AR glasses app communicate with the two databases to establish connections and support information exchange. Therefore, users needed a secure WiFi connection to use the system.
3.2. Setup of Wormhole endpoints
To explore the design aspect of location for a physically-grounded connection system, we designed Social Wormholes to enable users to create an ecosystem of connected artifacts and configure the physical locations for their connection endpoints, as shown in Figure 1. To create a wormhole connection, a user in a pair must first place the printed markers onto an artifact of their choice in their physical space. The user then initiates a connection in the setup app by taking a photo of their marker, and providing a text label to describe the connected artifact. Once saved, the incomplete connection will appear in a list to both partners in the app. The remote partner must then complete the connection in the setup app, by choosing the corresponding side of the incomplete connection, taking a photo of the marker on their chosen artifact and assigning it a label. In our technology probe, participants in each pair were given an opportunity to either freely place their markers, or coordinate with each other during the onboarding video conference call to set up their wormhole connections. Connected wormholes will appear in each user’s list of connections with an image and label (Figure 4). This process can be repeated for new wormhole connections. To edit the artifact of a wormhole endpoint, a user can simply move the printed marker to another artifact and update the label accordingly.
3.3. Pretesting and Usability Iterations
The team engaged in iterative prototyping and testing of the system. While a subset of the researchers focused on the system implementation, the others focused on testing it and providing feedback to improve the user experience. For instance, when pre-testing an early implementation of the system, we noted that some people had difficulty successfully scanning the paper markers with the AR glasses’ forward-facing camera. We therefore iterated on the user interface design and added four dots to help people center the markers correctly within the camera’s field of view for successful scanning. Our pre-testing also revealed that users sometimes struggled to understand when a transmission occured successfully. To improve this, we added animation effects to make this more apparent.
4. Methods
In order to gain insight into how people would leverage and behave around a ubiquitous constellation of distributed and physically grounded artifacts for social connection, we designed and conducted a two-week field study with pairs of remote friends in their respective homes using the Social Wormholes technology probe. Since this study was conducted within a technology company, it was reviewed and approved by the company’s compliance, legal, and privacy teams prior to being conducted to ensure that the subject matter and approach complied with ethical standards and that participants’ data was processed appropriately. In this section, we elaborate on the study procedure, measures, and approaches used for data collection and analysis.
4.1. Participants
| Pair Number | Gender | Age | Education Level | Ethnicity | Relationship (IOS Scale (Aron et al., 1992)) |
|---|---|---|---|---|---|
| P1a | Man | 36 | Bachelor’s Degree | White | 4 = equal overlap |
| P1b | Man | 37 | Bachelor’s Degree | South Asian | 3 = some overlap |
| P2a | Woman | 30 | Bachelor’s Degree | Not Reported | 2 = little overlap |
| P2b | Woman | 27 | Master’s Degree | East Asian | 2 = little overlap |
| P3a | Man | 28 | Master’s Degree | East Asian | 5 = strong overlap |
| P3b | Woman | 27 | Bachelor’s Degree | East Asian | 4 = equal overlap |
| P4a | Woman | 23 | Bachelor’s Degree | Hispanic | 4 = equal overlap |
| P4b | Woman | 27 | Bachelor’s Degree | White | 5 = strong overlap |
| P5a | Woman | 21 | High School | East Asian | 1 = no overlap |
| P5b | Woman | 21 | Bachelor’s Degree | East Asian | 1 = no overlap |
| P6a | Woman | 26 | Bachelor’s Degree | South Asian | 1 = no overlap |
| P6b | Woman | 30 | Master’s Degree | South Asian | 1 = no overlap |
| P7a | Man | 27 | Bachelor’s Degree | White | 5 = strong overlap |
| P7b | Man | 27 | Master’s Degree | White | 5 = strong overlap |
| P8a | Man | 31 | Master’s Degree | White | 7 = most overlap |
| P8b | Man | 31 | Bachelor’s Degree | White | 4 = equal overlap |
| P9a | Man | 32 | Bachelor’s Degree | White | 3 = some overlap |
| P9b | Man | 30 | Some College | White | 3 = some overlap |
| P10a | Man | 32 | Bachelor’s Degree | White | 4 = equal overlap |
| P10b | Man | 32 | Bachelor’s Degree | White | 3 = some overlap |
| P11a | Man | 29 | Master’s Degree | Hispanic | 4 = equal overlap |
| P11b | Woman | 30 | Bachelor’s Degree | East Asian | 7 = most overlap |
| P12a | Man | 30 | High School | Hispanic | 6 = very strong overlap |
| P12b | Man | 28 | Some College | African American | 6 = very strong overlap |
12 pairs of participants (24 people total, including 10 females and 14 males) completed the study in full (see Table 1) and were recruited from a technology company (name redacted) using an employee mailing list and internal communication channels. Since sample size for reaching data saturation cannot be predicted in advance, we determined our target sample size 10 2 following recommendations by experts for empirical grounded procedures in HCI (Caine, 2016).
The recruited participants were from a diverse set of teams within the company, representing a wide variety of technical and non-technical backgrounds including HR, sales/marketing and art/creativity. They were required to have access to WiFi in their homes and iOS smartphones with Bluetooth connection. Otherwise, there was no requirement for technical skills to participate in the study. Participants were located in 17 cities including Los Angeles, New York, London, San Francisco, Pittsburgh etc. Two pairs of participants, P9 and P11, were in different time zones from each other. Participants were recruited in pairs who considered themselves friends, except for two pairs, P5 and P6, that were formed ad-hoc and did not know each other before the study. Their ages ranged from 21 to 37 years (M = 29, SD = 4). Fifteen of them had a Bachelor’s degree, six had a Master’s degree, three completed some college, and two were high school graduates or equivalent. On a 5-point Likert scale (1 = Not at All, 5 = Extremely Frequently), participants on average reported not using AR/VR glasses at all (. Each participant was compensated with a $100 Amazon gift card for completing the study.
4.2. Procedure
Before the study, each participant was shipped a set of five printed wormhole endpoints, a pair of AR glasses, and instructions for setting up the glasses. On the first day of participation, two researchers remotely on-boarded participants in pairs during one hour long sessions conducted over video call. Researchers asked participants to fill out pre-survey questionnaires, guided them in setting up their systems, and and walked them through how to use Social Wormholes. The researchers answered all of the participants’ questions to ensure that they understood the system’s features and were comfortable using it independently. After the on-boarding session, participants received reference guides containing visual step-by-step explanations and instructions, which they could review when needed. Furthermore, the researchers created dedicated Slack channels for each participant pair. In this way, participants could contact the researchers for prompt assistance any time they had questions or concerns.
Participants were asked to consistently use Social Wormholes for at least 15 minutes per day for roughly two weeks. They were also asked to answer daily survey questionnaires that consisted of four open-ended questions about their usage through an online form. Participants used Wormholes for 11 days on average, with a maximum of 14 days. On the last day of the study, two researchers held an hour-long exit interview session with participants. Participants were interviewed in pairs. Finally, participants were also asked to complete an online exit survey. All surveys were issued via Google Forms.
The studies were conducted in phases, to ensure that any issues (e.g. technical, usability, procedural, etc.) could be identified and addressed without affecting all samples. We planned to exclude participant pair data if they were to encounter any major issues. Initially, we deployed Social Wormholes to two participant pairs in the first phase, followed by five pairs for each subsequent phase until we reached our target number of 12 pairs. Since no significant issues arose over the course of these phases, no data sets were discarded.
4.3. Measures
4.3.1. App Log Data
We recorded log data of participants’ usage of Social Wormholes to gain insight on a more micro-level regarding how participants used the app’s features. This includes information on markers — when they were installed and what names were given to them, and content — when were the ghosts and sparkles sent, which Wormhole endpoint they were sent from and sent to, and what content was transmitted. Data such as the Wormhole endpoint labels and the number of transmissions made per day were automatically logged by the phone-based setup app and the AR glasses connection app respectively.
4.3.2. Pre-Survey
During the onboarding sessions, each participant completed a pre-survey so we could have insight into their backgrounds and the quality of their relationships prior to experiencing the technology probe. This included questions on demographic information (i.e., age, gender, education level, ethnicity), previous AR/VR experience, level of loneliness using the short-form measure of loneliness (ULS-8) (Hays and DiMatteo, 1987), and the closeness of the partners’ relationship with each other using the single-item Inclusion of Other in the Self Scale (IOS Scale) (Aron et al., 1992). The ULS-8 (Hays and DiMatteo, 1987) contains 8 items (2 positively worded, which are reverse-coded), that should be rated on a 4-point scale (1 = Never 4 = Often). Examples of statements include “I lack companionship,” and “I am an outgoing person.” The IOS Scale (Aron et al., 1992) asks people to select a single option on a 7-point scale (1 = Not at all close, and 7 = Extremely close), ”Which picture best describes your relationship with your study partner?” A set of pictures with progressively overlapping circles, labeled “Self” and “Other” were provided to give a visual indication of the options available. See the supplemental materials for more details.
4.3.3. Daily Survey
We asked participants to fill out a daily survey (refer to the supplemental materials for example questions) during their participation, which comprised multiple choice, Likert, and open-response questions in a Google Form. We did this to capture the participants’ thoughts and rationale behind their interactions over the course of using the technology probe. Custom Likert scale questions based on specific features of the Social Wormholes technology probe were issued, such as: “Symmetric connections improved my sense of partner’s state over asymmetric connections,” (1 = Strongly Disagree, 7 = Strongly Agree). They were also asked open-ended questions about what types of activities they engaged in over the day, whether they modified their Wormhole connections, what they tried to transmit, and how they perceived transmissions from their remote friend (e.g. “What activities do you think your study partner engaged in while using Social Wormholes today?”). Participants were reminded to complete the survey via daily calendar events that were scheduled in negotiation with them. We periodically reviewed participants’ responses to the daily survey over the course of the study to inform our exit interviews and to verify that participants were completing the questionnaire in good faith. Given the time and effort involved in filling out a survey every day, we required participants to fill out a minimum of one daily survey per week.
4.3.4. Exit Survey
At the conclusion of the field study, participants were asked to each answer an exit survey (see supplemental materials), so we could gain insight on how the technology probe impacted the quality of their relationship, as well as learn how their setups may have changed over time and why. This exit questionnaire included questions that revisit their level of loneliness and relationship with their study partner using ULS-8 (Hays and DiMatteo, 1987) and IOS scales (Aron et al., 1992) to capture a post-intervention measure (see descriptions of these scales in Section 4.3.2 or refer to the supplemental materials for more detail), and a longer and more comprehensive set of closed-ended and open-ended questions on their setup and usage of the technology probe. Examples of questions include, “Where is Wormhole X, and how did you decide this?”, ”X affected where I placed my Wormhole markers.” One question also inquired about the sense of presence: “While using Social Wormholes, I can feel my study partner’s presence in my space (1 = Strongly agree, 7 = Strongly disagree).”
4.3.5. Exit Interview
To conclude the study, the researchers conducted semi-structured interviews with participant pairs over video conferencing calls to elicit rich insights into their thoughts, feelings, and perspectives regarding their engagement with the probe. Interviews were based loosely on the pre-prepared questions (see the supplemental materials). Unscripted follow-up questions were asked to dig deeper into points raised by participants. All interviews were recorded with participants’ consent and were later transcribed. Each interview lasted approximately 60 minutes.
| Asset | Total Count | % | Mean/Person (SD) | Mean Count/Person/Day (SD) |
|---|---|---|---|---|
| Transmissions | 2416 | |||
| Ghosts | 576 | 23.8% | 23.46 (14.01) | 1.89 (1.41) |
| Sparkles | 1840 | 76.2% | 75.33 (45.94) | 5.77 (2.95) |
| Connections | 54 | |||
| Asymmetrical | 39 | 72.2% | 3.93 (1.22) | |
| Symmetrical | 15 | 27.8% | 1.07 (1.22) |
4.4. Data Analysis
We collected a combination of quantitative and qualitative data using the aforementioned measures. We extracted total counts for connections, endpoints, and transmissions from the app log data, and we further aggregated and analyzed these metrics per participant pair (Table 2). Grand averages were computed from the IOS scale for relationship closeness, and from the ULS-8 scale for loneliness. Sense of presence were computed from the responses to survey questions (Table 3).
After transcribing the exit interviews, two researchers independently sectioned the transcripts into quotes for a bottom-up, open-coding approach to data analysis (Charmaz, 2006). Afterwards, the researchers worked together through multiple rounds of meetings to iterate on the codes, discuss their similarities and differences as part of a comparative analysis (Merriam and Tisdell, 2015), and leverage them in an affinity diagramming process (Holtzblatt and Beyer, 2017). The researchers determined that they reached code saturation when neither researcher could identify new codes or arrive at new interpretations of the existing codes after several rounds of revisiting the quotes. In accordance with Mcdonald et al. (McDonald et al., 2019), we did not compute inter-rater reliability (IRR), since we used the coding process to discover emergent themes or recurrent topics and permitted multiple possible interpretations of the meaning of the codes. After the two researchers completed their synthesis of an affinity diagram, two additional researchers reviewed the themes and provided their comments. The themes and sub-themes that emerged from this process highlighted an array of behaviors, perspectives, preferences, opportunities, and concerns, which we report on in Section 6.
5. Results: Social Wormholes Usage
24 participants (12 pairs) completed the study. One person went on a trip and had to re-establish the location of her wormholes during the course of the study. We logged 576 Ghosts and 1,840 Sparkles transmitted between participants, and collected 24 pre-survey responses, 118 daily survey responses, and 24 exit survey responses. On average, participants used the probe more than one session per day (M = 1.56, SD = 0.32) at different times of the day. All participants completed the pre-survey, exit survey, and exit interview. All participants completed between two and ten daily surveys, on average more than four surveys, meeting our minimum requirement of at least one survey per week.
We report on our findings of how people used the technology probe during the study, including what artifacts people assigned as connections, how they chose to map them together, and how they used the flexible number of connected artifacts.
5.1. General Usage and Self-reported Effects
Volume of Transmissions: Participants initially installed 129 wormholes with 47 types of artifacts (e.g. desk, kitchen, wall, bedroom), and in their exit survey responses they reported 125 wormholes with 25 types of artifacts. During the field study, 1,840 Sparkles and 576 Ghosts were transmitted. Participants on average sent 23.46 Ghosts (SD = 14.01) and 75.33 Sparkles (SD = 45.94) to their study partners throughout the study. Refer to Table 2 for a summary of these metrics.
Effects on Social Connection: The pre- and exit-survey scores suggest that the technology probe generally decreased their feelings of loneliness and increased their feelings of connectedness with their study partners. The mean ULS-8 loneliness score was lower for the post-measure (M = 1.65, SD = 0.51) than the pre-survey measure (M = 1.77, SD = 0.49). While both means correspond with a generally low level of loneliness, 20 participants’ loneliness scores decreased. A one-sided Wilcoxon signed-rank test on participants’ responses to the short-form measure of loneliness (ULS-8) (Hays and DiMatteo, 1987) before and after the technology probe (collected in pre-survey and exit survey respectively) shows a statistically significant decrease in participants’ feeling of loneliness, with Wilcoxon statistic . Regarding relationship closeness, the median scores from the IOS Scale for the pre- and post-measures were equivalent (Mdn = 4). However Through an additional Wilcoxon test on participants’ responses to the Inclusion of Other in the Self Scale (IOS Scale) (Aron et al., 1992) before and after using Wormholes, we also observed a statistically significant increase in their connectedness ratings, with Wilcoxon statistic . Statistical analysis shows that participants’ responses indicated they were significantly less lonely and had stronger connections to their study partners after participating in our study and using Wormholes. Data on these metrics are included in Table 3.
Effects on Sense of Presence: Alongside feelings of connection and loneliness, participants also indicated an increase in their sense of presence of their remote study partners in their physical space when using the technology probe. We observed a statistically significant correlation () between the number of transmissions (captured in the log data) and participants’ ratings to the exit survey 7-point Likert scale question “While using Social Wormholes, I can feel my study partner’s presence in my space.” (1 = Strongly Agree, 7 = Strongly Disagree). This suggests that participants who received more transmissions during the study reported a stronger sense of their study partner being with them. In the daily survey, P10b commented “I saw Sparkles from the desk area wormhole, which made me feel like they left their presence for me to discover.” Data on this metric is included in Table 3.
Impressions of AR and AR Glasses: As we describe in the beginning of Section 3, we chose an AR glasses-based approach for our technology probe because it gave us the properties that we needed to study how ubiquitous computing and a potentially large ecosystem of connected endpoints can foster social connection. The AR design itself was thus not a central focus of this investigation but more of a means to an end. Yet, because we used AR as a platform for this research, we were also able to glean insights about users’ attitudes toward staying interconnected via AR and AR glasses.
In exit interviews, several participants remarked that they appreciated how AR “just makes [communication] more alive, rather than a flat picture or a flat video” [P5a], compared to Snapchat, their usual means of staying connected with their study partner. This suggests that they saw value in rendering their communication itself in a more physical way, where the communication itself is manifested as a thing (an object).
Some participants felt that AR-based systems could lead to greater consequences for privacy since they require users to wear a camera or to point their phone’s camera around their home. In the case of wearable cameras, these participants were unclear about what exactly was in the camera’s field of view. We describe this and other privacy implications in Section 6.4.1.
| Pre | Post | ||||
|---|---|---|---|---|---|
| Average | SD | Average | SD | ||
| Relationship Closeness (IOS (Aron et al., 1992)) | Mdn = 4 | 1.86 | Mdn = 4 | 1.74 | |
|
M = 1.77 | 0.49 | M = 1.65 | 0.51 | |
| Sense of Presence | N/A | N/A | Mdn = 3.96 | 1.67 |
5.2. What Physical Artifacts Are Used?
During onboarding for the technology probe, participants were guided to choose a vast array of physical artifacts in their homes to configure as wormhole endpoints, spanning from small objects, furniture pieces to wall surfaces of a room. With this customizability, we want to understand what types of physical artifacts people choose to use to stay connected and how they use them for communication.
5.2.1. What artifacts did people use?
During set up based on log data, participants initially put their wormholes on 47 types of artifacts, with the top five most popular options being a desk (22), kitchen (11), fridge (6), wall (6), and door (6). In the exit survey, the participants were asked where the wormholes were by the end of the study and whether or not they had been moved. Wormholes converged to a smaller subset of 25 types of artifacts, the top six being a desk (29), kitchen (14), wall (9), bedroom (9), fridge (7), and door (7). Through the process of the study, participants have explored and thereafter discovered what artifacts they prefer to use as connections.
This suggests that many participants chose to configure wormholes on common household objects and spaces. During exit interviews, some participants shared that they connected artifacts that are very personal and special rather than commonly intended for communications. P3b, P4a&b and P10a centered their communication to their shared hobbies, such as playing piano [P3b, P10a] and sharing tarot cards.
“My favorite part was when I saw sparkles at his piano wormhole and heard him playing music — it made me so happy as he has been helping me look for a piano to begin lessons and I remember him telling me that when he plays piano he feels happy or relaxed, so I was also happy for him!” – P3b
P4a and P5b experimented with putting the wormhole on their pets or capturing playful moments of their pets222Participants confirmed that no animal was harmed during the study..
”I think it was pretty special when I sent you a ghost of my dog, which was also really hard to do because it kept moving around, and I just trying to track and follow them. ” –P4a
“I just waited until he [P5b’s cat] was laying down, and then the first time when I made the connection it worked, he was staying still and everything. Next time he was not friendly [laughter].” – P5b
5.2.2. How did people choose the artifacts?
From the Likert-scale question in the exit survey, “_ affected where I placed my wormhole markers. (1 = Strongly Disagree to 7 = Strongly Agree),” participants reported routine (), environmental lighting (), wormhole marker size (), and household occupants () to be the factors that mattered the most for determining the placement of the wormholes.
Many participants placed their wormholes in a manner that suited their daily routines. This suggests that people desire connection technology to be seamlessly integrated into their existing environments, around spaces in their homes in which they regularly frequent while following their normal routines. P5b, for instance, reported choosing artifacts specifically for connecting over everyday meals:
“It’s by my stove, since I also visit the kitchen frequently. I thought it would be nice to connect with my partner over the food we are eating.” – P5b
Artifacts that are located close to people’s routines tended to get used more often than others. P4b explained that he used the wormhole in his kitchen the most because lots of routines happen around it:
”It was just the one that I had the most to do around it. There were my coffee mugs there, there were my dishes, my food, my stove, so I thought about doing it more when I was at that one than any other one.” –P4b
As another factor affecting where they placed wormholes, participants considered how well the wormholes fit into their space (echoing two of the factors found in exit survey: the wormhole marker size and other household occupants). P11 explained in the exit interview for not wanting the wormholes to clutter her space, while P3 elaborated that she wanted to customize the wormholes to fit her home decoration:
“I just wish that instead of these markers, I had a cute little object that was unobtrusive and looked like a decoration that would trigger these things instead of a wormhole, because it doesn’t go with the rest of my vibe in my house.” – P3b
Additionally, one pair of participants [P7a & b] commented that they would choose artifacts based on special occasions. When they saw interesting things outside their residence, they wished there was a wormhole to share it. P7a brought the wormhole to places outside their routine so that they can capture interesting things there, such as pool and baseball games. One participant, P2a, selected an unusual and awkward artifact for the sole purpose of making a joke:
I also put it in funny places because I think it should be a light-hearted thing…I put it up with toilet where you sit on the toilet and it’s on the wall right next to you. It’s really funny. - P2a
5.3. How are Artifacts Connected to One Another?
Social connection systems are often designed to be symmetrical, meaning that identical artifacts are used as a point of connection (e.g. A’s candle is connected to B’s candle). In this technology probe, participants chose to map things symmetrically and asymmetrically (i.e. when the artifacts are different, such as a patio table to car). We report on how participants configured their connections to elucidate how people draw interpretations from social connections.
5.3.1. Symmetric vs. Asymmetric Connections
Participants established 54 wormholes connections (15 symmetrical, 39 asymmetrical) in total with our technology probe over the duration of the field study. These metrics are summarized in Table 2. Participants indicated in the exit survey that symmetric connections improved the sense of presence with their partners, and made it easier to understand their partner’s state and share their own state. Asymmetric connections led to special experiences.
In our daily surveys, participants agreed (rating of 4 or higher) with the statement “Symmetric connections improved my sense of partner’s state over asymmetric connections,” 75.3% of the time (); and agreed (rating of 4 or higher) with the statement “Symmetric connections make it easier to share my state over asymmetric connections,” 80.4% of the time (). For example, P9a&b mentioned connecting their refrigerators to share cooking moments with each other, while P12a&b described how symmetrical connections at their desks made their communication experience more immersive and helped them get a sense of working together.
Participants made many different types of asymmetrical connections including connections between one participant’s TV [P2a] their partner’s fridge [P2b], between one participant’s plant [P2a] to their partner’s toilet [P2b]. In addition, P2b kept all of their wormholes on their desk, while P2a scattered their wormholes throughout their house—their desk, TV, the wall next to their desk, their shower wall, and their car. While asymmetric connections might not be easier for participants to understand and share with each other, participants reported that the asymmetry produced a special, interesting experience:
”I do really enjoy asymmetrical experiences because my partner would have her wormhole on her cat for example, which will spice up the experience as a whole and keep it interesting.” – P5b
5.3.2. How Connected Artifacts are Interpreted by Users
We synthesize from daily surveys, exit surveys, and exit interviews that participants saw different degrees of value in the location of their artifacts. Some appeared to have a more spatial understanding of their connections; others focused on connecting over shared activities.
Participants with strong spatial interpretation likened the experience to having a sense of presence [P10b] or leaving notes for each other around the house [P6a]. In daily surveys, participants reported that knowing where the transmissions are coming from adds significant value in understanding their partners’ activities and daily routines.
“I would say it was very unique. It reminds me of my parents would leave notes around the house when they were not there. You’re not expecting it and then you stumble on it.” – P6a
Others, rather than placing significance on where their artifacts were located, simply saw the connections as a direct channel of information exchange to their partner. Participants rated the statement “I felt that it was valuable to receive transmissions at particular places.” in the exit survey with an overall neutral score (). For example, P11 stated that she prefers a direct form of communication and would like the wormhole to be “partner-based” instead of “object-based”:
“It will be great if they can be customized, and I decide that in my room, this is [P11b]’s wormhole, and then I can glance at it any day I want, just to see her messages to me. ” – P11a
Still others interpreted the wormholes connections as their partner’s state of activity. 29 daily survey responses reported using wormholes to share activities. Our participants experimented with using Wormholes when working (reported 15 times in daily surveys), eating (14), doing chores (13), cooking (11), walking around (10), getting ready in the morning (8), watching TV (6), playing music (5), organizing/trying on clothes (5), driving (2), and doing laundry (1). In exit interviews, P10b stated that he frequently used wormholes to signify his work status. He tried to communicate a break in his work day - making coffee, singing to music, looking in the fridge. P2a reported working at his desk and transmitting a Ghost of his keyboard and mouse. P12a enjoyed connecting with their partner while working and experienced a strong sense of working together: ”I can imagine himself typing when I’m typing” -P12a .
Through a sequence of asynchronous transmitted artifacts, some users were able to interpret a sequence of transmissions as a continuous line of daily activities from their partner:
“I saw Sparkles coming from their door and their coffee machine, so I figured she must be starting her day; I saw lots of Sparkles by the kitchen and door, which made me think she was cooking before she left. ” – P4a
P7b and P5a stated they used the wormholes to track routine activities throughout the day, wherever they went:
I’m at my desk, I’m drinking water. I said, I’m ghosting my water bottle or I just finished washing my face […] I sent a my skincare or even my fan when it’s hot outside of summer. […] Oh yes, and then also washing the dishes. – P5a
5.4. How Many Artifacts Are Used?
Social Wormholes system log data and daily surveys show that 23 out of 24 participants who completed the study used more than 4 wormholes everyday. Most participants preferred to have large quantity of distributed connection endpoints. In exit interviews, participants reflected that the distributedness provides better coverage and a more diverse communication experience with their friends:
“[With] a greater number, I could definitely explore more options and get a more, a larger variety of contents. Just to learn more about like where she is at a moment versus having fewer. If we did have fewer wormholes, it might just be the same few things that are happening.” – P5b
P5 and P6 also mentioned the distributed wormholes made the experience of maintaining awareness with their study partner feel like a “scavenger hunt” in a good way:
“It was a treasure hunt of like, ‘Oh, where will I see it?’ That experience was really cool.” – P6a
Large number of wormholes also support spur-of-the-moment sharing and encourage participants to use wormholes for a variety of reasons. 31 comments from the daily surveys indicated that participants were motivated to use wormholes when they were physically around the wormhole location.
On contrary, other participants mentioned in exit interviews that having a large number of wormholes might make it inconvenient for them to check messages. Some preferred using only one wormhole or a centralized location for all wormholes. According to the log data, P12a mainly used one wormhole endpoint throughout the user study (i.e. used one of their endpoints more than 80 times, and used others less than 5 times). In the exit survey and interview, P11b shared how she compiled all wormholes together for ease of use:
“ I think in the beginning, we had [the wormholes] placed in different corners and I eventually just consolidated them. I had them in the pile near my desk. I always go through them in order.” – P11b
6. Results: Themes and Design Implications
Following exit interviews open coding and affinity diagram analysis (see Section 4.4), we report on the set of themes of people’s behaviors and resulting preferences when given the power to create their own ecosystems of connected objects for social connection, which include people’s broader patterns of behavior for using ubiquitous, physically-grounded social connections throughout their day (Section 6.1) and what each of their connection endpoints represents to them (Section 6.2). We summarize our results and present our design implications for future physically distributed social systems in Table 4. We also reveal how people adopt a mosaic of behaviors around how they use these social connections (Section 6.3), and other important social considerations for these social connections such as privacy (Section 6.4).
| Theme | Insight | Design Implications |
|---|---|---|
| Ritualistic Communication (Section 6.1.1) | Users integrate physically-grounded communication in conjunction with daily routines and seek efficient and stable means of sending and receiving messages. | Keep records of the artifacts along the user’s routine, recognize changes in the user’s routine over time and realign connected artifacts to their daily lives. Provide a centralized channel to recover messages for a sense of control and reassurance. |
| Serendipitous Communication (Section 6.1.2) | Users favor connections that are asymmetrical and changing because they add a sense of surprise to communication. | Future systems can prompt users with creative ideas for connection points and mappings. Expand connections to include one-to-many, many-to-one, and many-to-many connections. |
| Artifacts as Proxies for People (Section 6.2.1) | Some connected artifacts are interpreted as proxies for a person’s presence in ones space, similar to ”shrines.” | Future systems can keep records of the relationships between the user and their friends, offering suggestions for artifacts and spaces that suit the nature of the relationship. |
| Some connected artifacts are interpreted as direct channels to their partners, similar to a 1:1 chat. Users value instant updates from these, and they wish to carry the endpoints around as wearable devices. | Sparkles and Ghosts can allow users to share their location in a less intrusive way than GPS or other means. Partners wishing to share their location in a low fidelity way may embrace the usage of Ghosts to give context. | |
| Artifacts as Overlapping or Connected Spaces (Section 6.2.2) | Some connected artifacts are interpreted as means of co-habiting spaces with partners for a sense of presence. Users prefer intuitive symmetrical mapping for this purpose. | Recognize similarity between physical spaces and suggest symmetrical connections. Map the spatial co-relationship between the transmitted object and the wormhole and enable a representation of the transmitted object placed in the environment with spatial mirroring. |
| Artifacts as Shared Activities (Section 6.2.3) | Some connected artifacts are used as channels for sharing particular activities. Users interpret interactions with the wormhole as signals that the partner is doing the activity. | Future systems can identify common activities that both connected partners engage in and suggest activity-signifying artifacts and spaces as connection points. |
6.1. Behavior Patterns Around Communication
Based on users’ self-reports, descriptions, and comments of their usage from exit interviews, we discover two clusters of behavioral patterns toward exploring the technology probe as a communication medium: ritualistic and serendipitous. A ritualistic behavioral pattern leads users to seek efficient and stable means of sending and receiving messages. A serendipitous behavioral pattern motivates users to be experimental and creative about their process of connecting with others.
6.1.1. Ritualistic Communication
Some participants [P4a, P5a&b, P9a&b, P10b, P12a&b] establish a predictable and ritualistic pattern in their daily communication. They placed their wormholes on artifacts they were using everyday and integrated the physically-grounded communication in conjunction with routine moments of their life, such as placing one on their bedroom mirror to share their outfits in the morning or on their coffee machine to share information about their breaks.
Both P4 and P9 reported sharing outfits of the day with their study partners habitually. P4a described when she was getting ready, she sent her outfit to her partner through the wormhole on the mirror, to show what she was wearing that day. P4b described her enjoyment receiving her study partner’s daily update of her outfit: ”I really enjoyed the Ghosts that she sent with her mirror because I could tell that she was having fun there. ” – P4b
Participants P5a and P7b described in detail how they shared a constant sequence of their routines throughout the day (Section 5.3.2). Participant pairs P5 and P9 developed a closer relationship with their partners by learning about their routine activities and environments:
”He has a bit of like routine that I didn’t know. I thought he just works and then goes and falls asleep until he comes back to work. I found out that he has a lot of the cooking routines and some of the stuff he would send me, just you can see a pattern.” - P9b
In ritualistic communication, users’ mappings were usually symmetrical, which made it intuitive for them to interpret the activity or space of their partners. To maintain a predictable pattern, they often desired to receive all the messages at expected places. This meant that they often only use a small set of fixed connections so that they would not miss out on any messages [P9b, P12b].
”For me, I would have preferred three or less. I think it would have just been more convenient and easier to keep up.” - P12b
Future systems can support a predictable and ritualistic communication experience by being designed to keep a long-term record to the connected artifacts along the user’s routine. The system may also learn about the user’s routine and recognize any small change of habits over time, in order to help users realign the connection points to their daily lives. Since symmetrical mapping is most intuitive for connecting partners, the system should prompt the users to establish symmetry during set up and maintenance. To avoid missing messages, users would appreciate a notification or a centralized channel to recover messages for a sense of control and reassurance.
6.1.2. Serendipitous Communication
Several participants [P2a, P3a, P4a&b, P7a&b, P10a, P11a] exhibited a serendipitous tendency towards how they used Social Wormholes. Participants also displayed serendipity by entertaining their partners with hobbies [P3b, P4a&b], playing with pets[P4b, P5a], making jokes[P2a], or sharing interesting occasions and events[P7a&b]. People behaving serendipitously found it important to incorporate some novel artifacts or something else that is different from the day before. They struggle to see the value of transmitting a routine object.
”I found a hard time to everyday find one single object that I would want to send to [11b]… maybe if I got a new pair of shoes and I want to send the one shoe to her.” - P11a
Users with serendipitous tendencies experimented with unconventional artifacts (i.e. non-standard household objects) as connection points. Examples include moving objects, pets, locations that provoke humor, and artifacts related to personal hobbies. The mapping of connections were often asymmetrical and changing, as the users preferred to spice things up, keep things fresh, and react to momentary changes of events. For both sending and receiving, users embraced a large number of connections so that they could share wherever and whatever they wanted and so that they could have the treasure hunt-like experience of finding messages by surprise [P6a].
Future systems can support and encourage serendipitous communication by adopting high customizability and flexibility in terms of what artifacts can be activated as connection points. The system should instigate the discovery of potential creative wormholes and transient changes in mapping connections. Changes in connected artifacts and mapping should be tracked and streamlined by the system rather than burdening users to provide accurate labels. Since a greater number of connection points leads to more spontaneous and surprising sending and receiving, future systems could multiply the amount of surprising opportunities by expanding to include one-to-many, many-to-one and many-to-many connections.
6.2. What Physical Artifacts Represent
Learning from how artifacts are connected and interpreted by users, as well as users’ comments and reflections, we notice that people configure physically-grounded connections with different ideas for what they represent. Artifacts could be interpreted in three ways: as a proxy for a person, as a means to connect a user’s space with their partner’s space, or as a means to share an activity with their partner.
6.2.1. Artifacts as Proxies for People
From their comments in exit interviews, we notice that several users [P2b, P4b, P7a & 7b, P11a] considered their connected artifacts to be direct channels to—or proxies for—their partner.
This behavioral pattern parallels the concept of reaching someone with a smartphone or landline phone. In this paradigm, the location of a partner’s phone is not necessarily important; rather, the fact that a person can be reached through it as a medium takes precedence. Similarly, artifacts could serve as a proxy for a person by substituting for another person’s presence in a space. P4b and their partner P4a both lived alone, but they interpreted the wormholes as representations of their partner in their respective spaces, like ”having a low-impact roommate” - P4b. While some participants stated one proxy was enough [P7a, P9b], others accepted the idea of having many proxies spread out across multiple locations in their homes. In this way, they could even provide a tour of their house [P10a]. In yet another case, some people began to consider that even the content that was transmitted (and not just the wormhole itself) were proxies for their partners. For example, P7a and P7b came to associate the transmitted ghosts of their favorite beverages as stand-ins for each other.
We noticed that this general viewpoint for wormholes served two purposes: to act as ”shrines” and to act as wearable connections.
Regarding ”shrines”, people treating wormholes as proxies for other people found it important to mark a personal artifact of significance as a connection endpoint (e.g. a special figurine that signifies their friend) [P3b]. This mirrors former designs in Ambient Media that have a sentimental or personal nature (Chang et al., 2001; Heshmat et al., 2020). For receivers, the location to where the physically grounded receiving endpoints are placed in their own environment is rather significant. The placement speaks to the relationship with the person. For an intimate loved one, the receiver might place its artifact proxy in close proximity and private location, such as a nightstand. For a colleague, the receiver might place its proxy at a location that mirrors their relationship, such as a desk.
Future systems could support this perspective by being aware of the relationships between the user and their friends. Current social media has developed relationships classifiers in the social network; for example, Facebook includes groups of friends based on closeness and how they met. This kind of relationship network can be extended to physically-grounded connections by mapping people’s relationships to physical artifacts and spaces. With this knowledge, future systems could assist users maintain their connected physical environment meaningfully and orderly based on their personal relationship network.
Regarding wormholes acting as wearable connections, some people treating wormholes as proxies for other people expressed a desire for a wearable wormhole [P7b] or one they can carry around in their wallets so they could easily send and receive content from their partner instantly. As the connected artifact is kept on the user’s person, its relative position with respect to environment is not significant. However, they might still be interested to keep track of how their partner’s location changes throughout the day.
Future systems could design around the ability to get instant updates about their partner’s location. For instance, if the receiver is using one wearable wormhole and the sender is using multiple location-based wormholes, then the sender’s many endpoints can be used as a light-weight form of location tracker. The receiver can be notified on their wearable wormhole whenever the partner moves to a different location. Compared to a GPS location tracker, this format gives the sender a great degree of control over what locations are being tracked and shared with others.
6.2.2. Artifacts to Create Overlapping or Connected Spaces
Some participants [P4a, P5a, P6b, P9a&b, P10b, P12a] considered their connected artifacts to be a means of connecting their spaces with their partner’s spaces, similar to the literal concept of a wormhole. By establishing wormholes with this viewpoint, they gained a sense of presence of their partner [P5a, P9a&b, P10b] and appreciated that having connected artifacts could make it as though their friend can co-inhabit the space with them [P5a, P10b]. Some people also associate connected artifacts with the concept of leaving notes for each other around the house [P9a, P6a].
For these users, being able to choose multiple artifacts that can be seamlessly integrated into different parts of their homes is helpful. The specific artifact that is chosen is less important than how the artifacts are positioned within the connected spaces. P4a and P9a commented on the desire to have a greater degree of spatial co-relationship, such that they can position transmitted objects in their partner’s space mirroring positions in their own space.
[M]ake it a little bit more immersive and to feel like I’m in her space, I think it would have been cooler if it was more of a 3D image and if it was a specific place, especially if it could be wherever it was in relation to her wormhole. If she has her coffeemaker that’s off to the right of her wormhole, so she takes that photo, I’d want to have to look at the same place to see where is her coffeemaker in relation to my wormhole. - P4a
Symmetrical mapping of spaces is most intuitive and works best to support a sense of presence. Future systems could incorporate computer vision technology that plays a role in recognizing the similarity between physical spaces of both parties and makes suggestions to how spaces should be connected. Future systems could also use computer vision data to accurately map the spatial co-relationship between transmitted object and the connected artifact, enabling a 3D hologram representation of the transmitted object placed in the connected environment with meaningful spatial mirroring.
Additionally, the number and placement of connected artifacts may have an effect on the sense of presence. Future systems may optimize by suggesting centrally located artifacts in a space as connection points, such that more surrounding objects can relate to them spatially. The number of connected artifacts could be scaled based on the system’s understanding of the size of the user’s environment and the spatial coverage of each connection point. An optimized number of connections would help achieve a symmetrical spatial coverage between connected spaces and therefore maximize sense of presence.
6.2.3. Artifacts for Sharing an Activity
Several participants [P4a, P5a&b, P7a&b, P9b, P10a&b, P12a] considered their connected artifacts to be places in which to participate in a shared activity with their partner, aligning with many prior works on sharing activities over distance (Barden et al., 2012; Chai et al., 2017; Nawahdah and Inoue, 2013), in which cooking and dining are popular themes. By employing symmetrical connections, these participants vividly perceived themselves sharing activities such as cooking [P5b] or working [P12a] together. Participants also dedicated wormholes for the sole purpose of sharing special hobbies (piano [P4a, P10a]) and events (pool party and baseball game [P7a&b]).
Many participants enjoyed connecting asynchronously through activities. They interpreted a stream of transmissions from their partner’s different wormholes as continuous daily activities [P4a], which gave them a strong sense of co-presence with their partner. Building on this, P9b desired to have sequencing and grouping features to better illustrate how their partner transitioned between activities.
”I would like to see it as a story of five different ghosts, in order of which they were transmitted back to back. From my perception, that would give me a sense of seeing their life together […] If you tell the system what you are doing, then it can group those objects by story. If I’m cooking then I want send those multiple things from my kitchen, groups them together. ”- P9b
Interestingly, our technology probe was designed to support asynchronous connections, and yet, some users gravitated to having synchronous experiences, underlining the significance of supporting synchronous activities while maintaining the poetic sense of asynchronous ambient connections.
”…suddenly I knew we were both live in the moment together, that would have been maybe meaningful where we could connect, but there was really no way to align. Are we two ships sailing passing in the night or are we having a moment because we’re here in the moment together?” - P10a
Future systems for supporting the notion of shared, connected activities should identify which common activities both connected partners engage in and suggest a symmetrical mapping of connection points in the vicinity of that activity to the users. Future systems could identify activity-signifying artifacts (e.g.piano) as its suggestions. There is also opportunity to recognize the context of the activity in order to determine whether users would desire a synchronous or asynchronous sharing experience. Last, some activities may benefit from one-to-many broadcasting or many-to-many connections. A multi-location alumni reunion is a good example of such an activity. Future systems could explore how to support such synchronous and scalable connections, and how to mitigate multiple sequences of activities in a network of physically-grounded connections.
6.3. Mosaic of Behavior Patterns and What Connected Artifacts Represent
Emerging naturally from using the customizable technology probe, our findings reveal that people adopt multiple behavioral patterns and find values in all of them simultaneously.
Some participants predominantly displayed a ritualistic behavior pattern, but also enjoyed rare moments of serendipitous playfulness. P5b, as an example, found value in learning routine details about their partners daily life, but also took a spur-of-the-moment experiment using their cat as a connected wormhole. In some cases, the two partners in participant pairs adopted opposing communication patterns while connected in the same ubiquitous ecosystem. For example, P10a is serendipitous, while their partner P10b is ritualistic.
Participants also adopted different viewpoints for what each of their connected artifacts represent. For example, P7a established some connected artifacts acting as proxies of their partner and other connected artifacts acting as places from which to share an activity. Their favorite beverage served as a proxy of the partner on a daily basis, but during an interesting event such as pool party, they dedicated a wormhole to being a way to transmitting activities taking place during the event.
Given this behavior, we think that there is not one superior behavior pattern that physically grounded social connection technologies should cater to. Rather, we anticipate that many of these different perspectives can co-exist in people’s lives and in ubiquitous computing as a mosaic of different types of connected artifacts.
6.4. Other Social Considerations
While we focused on patterns of behavior that people exhibited, users also brought up considerations that are important to the design of future ubiquitous computing systems for social computing.
6.4.1. Privacy
Generally, people were comfortable sharing with one another since they were friends, but comments regarding privacy did arise [P6a, P9b, P10 a&b]. Some users hesitated or had questions about what the glasses captured and what we as researchers would be able to see. Future systems will likely need to clarify how privacy is maintained and protected by companies providing such services. We also anticipate that as the number of connected artifacts scales upwards, people may lose oversight of what objects are connected and sharing information. Furthermore, while our technology probe used printed markers, it’s possible that future systems can invisibly tag artifacts. This can exacerbate this potential issue. One possible solution for future investigation is establishing tools or methods for automatically curating and reviewing the information that is being shared through a person’s network of connected artifacts. Tools that can give users an overview of all of their connections may be particularly useful.
6.4.2. Overwhelming v.s. Loneliness
Despite there being more platforms than ever for online social connection, there is nevertheless a growing epidemic of loneliness. Furthermore, there is a problem around having too many communication channels demanding our attention and effort. These negative effects could extend into the realm of physically-grounded and connected artifacts, as one participant had speculated:
I feel like too many may feel a little overwhelming and intrusive. Then it turns into more of an inconvenient scavenger hunt, because yes, you constantly have to go around and look at a zillion different little things that you scan is or like, it could give you a bad reaction of loneliness, if you don’t receive anything and you have a zillion different wormholes and you don’t get anything. I feel like keeping it semi-small is maybe better. - P9b
To address this issue, future systems might play a more active role in curating the number of available connections. They may be able to make suggestions to users and/or automatically increase or prune unused points of connection to optimize in accordance with users’ needs and usage levels. Future systems might position themselves as streamlining and unifying communication rather than adding even more channels. As an example, it could be possible to surface a photo or video message from a friend sent on any communication channel (text, email, Slack, etc.) as a Ghost in a relevant location, then automatically forward any Sparkle or Ghost sent in response to whichever channel the original message came from.
7. Discussion
Our study had several key takeaways, including that users engage in several different behavior patterns for connecting objects and communicating through them (Sections 5.2 and 6.1), that users assign different roles to physical artifacts (Sections 5.3 and 6.2), and that each user may adopt these varied patterns simultaneously (Section 6.3). Table 4 expands on these findings in more detail.
Our investigative approach was unique from prior works in two ways. First, the vast majority of prior studies on the use of tangible or ambient systems for social connection have comprised a single pair or set of near-identical connected devices between remote parties (Siio et al., 2002; Dey and de Guzman, 2006; Chang et al., 2001; Kim et al., 2015). In contrast, by creating and deploying Social Wormholes, we gained knowledge into how ubiquitous computing—namely, an ecosystem of multiple artifacts— can be used. As expected, we found that ecosystems of multiple endpoints improve social connection and a sense of presence (see Section 5.1), similar to the range of single-connection systems. However, by studying the use of multiple lines of connection in concert with one another, we were in addition able to outline and distinguish between different patterns for social connection, as well as gain novel insight into how these patterns can co-exist with one another. Second, prior social connection systems have been designed based on researchers’ own design intuitions or through co-design processes with users (Dey and de Guzman, 2006). In other words, prior system configurations have largely been determined by researchers. Social Wormholes, instead, empowered pairs of friends to design and curate their own set of existing objects as endpoints for social connection on their own. This allowed us to learn what other objects and configurations may be fruitful, beyond those that have been previously identified, based on the usage patterns that emerged organically over the course of our two-week field study.
Stemming from our results, we discuss and contribute five points of knowledge to the prior literature regarding ubiquitous social connection systems.
First, we discuss that users value choosing their own artifacts. Designers may not always choose the artifacts that users themselves would and users would value this autonomy. With Social Wormholes enabling this ability, we found that several of the objects and places participants chose overlapped with those from prior literature (e.g. kitchen (Chai et al., 2017), bedroom (Siio et al., 2002), and mirrors (Schmeer and Baffi, 2010; Dey and de Guzman, 2006)) (see Section 5.2), thereby reinforcing that these are desirable to be used as endpoints for social connection. Furthermore, while our participants did not choose objects such as picture frames (Chang et al., 2001; Kim et al., 2015), some of their comments underlined the appeal of using personal tokens that are specially representative of their remote friend (see Section 6.2.1). On the other hand, there were artifacts that our participants chose to connect that have not been used before in prior systems, such as toilets for humor and pets. These choices corresponded more with the desire to share fresh content (see Section 6.1.2). Ultimately, more study data is needed to capture the full extent of artifacts that would or would not be ideal for social connection. Nevertheless, future ubiquitous systems should therefore give users more autonomy to choose and update their own artifacts, not only to account for user’s unique preferences and circumstances but also to support changing trends for artifacts, since users’ ideas of what artifacts are interesting or fresh can change over time.
Second, we discuss that the surrounding context of routines and other people in the space shapes users’ choice of artifacts. Routine was an important factor for users when choosing their artifacts (see Section 5.2.2). This strongly aligns with Cao et al.’s (Cao et al., 2010) highlighting of routine as a theme in remote communication. However, while some users allowed transmissions to occur as an accompaniment to their daily activities (see Section 6.1.1), others made it a deliberate and standalone practice to send transmissions (see Section 6.1.2). We also discovered that users are sensitive to how their choice of artifacts may affect or be perceived by other members of their household. To the best of our knowledge, our findings in this area are novel since most previous studies have focused on single users (Grivas, 2006; Häkkilä et al., 2018; Heshmat et al., 2020; Kim et al., 2015; Siio et al., 2002) or family units (Judge et al., 2011, 2010) rather than studying the system in the context of a mixture of both users and non-users in the same household. Future designers should therefore anticipate that routines and other people, both users and non-users, are contextual elements that will shape users’ choices and behaviors.
Third, we discuss the value of spatial context based on users’ perspectives. Spatial information is most valuable to people who seek to connect through overlapping spaces (see Section 6.2.2) or through shared activities (see Section 6.2.3). Participants expressed a desire for transmissions to be presented in true spatial relationship to endpoints, expanding the perceived footprint of shared space and making transmissions more real and immersive. This echoes prior works that stress the importance of spatial coherency (Kim et al., 2017) and the preservation of physical-space information in virtually-mediated exchanges (Fender and Holz, 2022). When connecting over a shared activity, the spatial information is valuable for adding a greater feeling of commonality and relation during an activity. For instance, participants enjoyed the illusion of their friend typing at their desk just as they were typing at their own desk. The spatial information adds an extra layer of social awareness (Sereno et al., 2020; Hassenzahl et al., 2012). In contrast, when users center their perspective on the artifacts as proxies for people(see Section 6.2.1), they value the space as a context to the closeness of their relationship, and additionally, they value timing context of the messages. Thus, future designs should cater to capturing and relaying spatial information to enhance the user experience based on their perspectives.
Fourth, our findings add to prior literature on the preference around asymmetric spatial connections (Voida et al., 2008), in addition to symmetric connections. In contrast to the common assumption that people would prefer symmetrical pairings, a surprising 72.2% of connections with Social Wormholes were asymmetrical (see Section 5.3). Prior works on tangible systems for social connection have often been designed in symmetric pairs (Li et al., 2018), placing heavy assumptions that symmetric connections are preferred. Our investigation directly studies how users choose to connect artifacts to one another and reveals the comparison between symmetric and asymmetric connections.
We find asymmetric connections offer three advantages. First, asymmetric connections do not require effort to coordinate artifact pairs between casual friends. Depending on the closeness of their relationships, close friends and romantic partners may associate their relationships with a spatial context (see Section 6.2.1) and therefore choose to coordinate symmetric connections. Second, asymmetric connections can afford a serendipitous and fun experience, when users receive surprising messages at unexpected locations (see Section 6.1.2). Third, asymmetric connections can broadly support the mosaic of perspectives, as we find that the two partners in a connection pair often adopted opposing communication patterns (see Section 6.3).
Lastly, we discuss that distributed endpoints encourage opportunistic sharing and ambient receiving. From the sending perspective, distributed endpoints provide greater coverage around users’ space and daily routine, making it easier to immediately share moments about their lives. As such, Social Wormholes worked well to encourage sharing of more routine or “mundane” activities (Tolmie et al., 2002) during “empty moments” (Lottridge et al., 2009) in their everyday lives. From the receiving perspective, the distribution of endpoints enabled people to receive messages when their time and place were relevant, which catered well to people who enjoy ambient receiving. These users focus on going about their daily life and perceive social communication as an accompaniment in the background space (Mankoff et al., 2003). Depending on the users’ individual perspectives, the wide spatial distribution however also compromises the desire for instancy. Having to receive the message at the designated location introduced a potential time delay and required users to physically arrive at the corresponding endpoint to receive transmissions. In turn, it made some users feel that it was more difficult to “keep up” with all of the information (see Section 6.1.1), potentially leading to feelings of overwhelm (see Section 6.4.2). Future designers should consider when it might be helpful to limit the number of connections and/or incorporate notification features into a ubiquitous social connection system to strike a balance between ease of use for sharing and feelings of control in receiving information.
8. Limitations and Future Work
Our study participants generally followed a work-from-home routine due to the COVID-19 pandemic, so our results apply to connection endpoints being used in this context. This unusual circumstance may have biased the participants’ towards more routine patterns of behavior, since people were encouraged to stay at home. In the future, it would be interesting to investigate ubiquitous connection endpoints in other contexts. Experimenting within professional office settings or public outdoor events, for instance, could yield additional insights into unique behavior patterns.
We recruited our participants from within a technology company, since the AR glasses hardware at the time of this study could not be distributed to the general public. Although we recruited participants with both technical and non-technical roles in the company, they were likely more interested in technology than average, and may therefore have been more biased in favor of the experience overall.
Our research is limited in that participants were suggested to use the AR glasses for a minimum of 15 minutes per day. This was to help ensure that each person would be able to try out the technology long enough to understand the features and see how it might be useful or create friction between them and their remote partner. We also wished to uncover knowledge about how usage may change over a duration of time, so we requested participants to use the technology each day throughout the two-week field study duration. As this may have influenced our findings, it would be valuable to revisit this topic in the future when wearing AR glasses regularly becomes more commonplace. Due to the COVID-19 pandemic and the widespread locations of participants, we were also unable to conduct real-world observations of participants using the technology probe that could have added richness to our findings.
In our study, we restricted artifacts to only be connected in a one-to-one fashion. While we used one-to-one connections as an initial starting point, it would be interesting to explore one-to-many or many-to-many connections between augmented physical artifacts, as mentioned in our design implications - particularly for serendipitous communication. Hence, there is an opportunity for future work to investigate how an ecosystem of connected physical artifacts can be used to nurture a person’s social connection to a larger network of friends.
Additionally, our study focused on one-to-one communication between two friends, primarily due to a limited number of hardware units available to support our study. Nevertheless, some participants brought up how sharing their living space with others influenced how they used Social Wormholes. For example, some participants mentioned that they were mindful of others in their households when deciding where to place their markers. Furthermore, we live in an era in which people do not only share their home environments, but also, adding to the complexity, face the demand of juggling multiple roles, such as being a parent, a friend and a colleague. In this case, communication technologies play a pivotal role in helping people maintain a network with others as supportive scaffolding in their dynamic home environments (Beckman and Mazmanian, [n.d.]). We suspect that the behavior patterns and perspectives we observed in this study may extend to these more complex communication scenarios – in particular, the tendency for people to engage in a mosaic of these. Exploring these themes in the context of negotiating over shared spaces and managing the division of roles would be an interesting line of exploration for future research.
We also see the opportunity for existing social applications to extend their reach beyond the digital domain and adopt an aspect of being physically grounded. Social networks could explore how synchronous and asynchronous updates about friends’ lives can be shown through physical artifacts in someone’s environment. For example, while existing gaming platforms often notify users when their friends start playing a game, such information could be anchored to a physical gaming console. Notifications could also be displayed with multimodal and tangible media. If a sender is near a coffeemaker wormhole, for example, the receiver might be notified by a pleasant scent of coffee. Furthermore, the present study focused on supporting social connections between friends, but it is also possible for updates from non-human entities (e.g. pets, companies, world news) to have a physical footprint in the home via associations with different artifacts.
9. Conclusion
As the potential for computing to blend into our everyday environments grows, powerful opportunities open up for social connection. Inspired by the vision of ubiquitous computing, this paper explored how distributed and physically-grounded social connections can be used to help family and friends stay connected. We implemented Social Wormholes, an AR-based technology probe that enabled pairs of friends to craft an ecosystem of connected physical artifacts through which to transmit information. The probe allowed users to choose what artifacts became endpoints for connection, how they were mapped to one another, and how many endpoints they wanted at any given time. Based on logs, surveys and interview data collected in a field study with 24 participants, we uncovered different perspectives and patterns of behavior for engaging with such a system. This includes ritualistic versus serendipitous behavior patterns as well as differing perspectives around whether these physical artifacts serve as a proxies for friends, as overlapping spaces, or as opportunities to connect over activities. Furthermore, we found that while different patterns of behaviors exist, users often adopt a personalized mosaic of these patterns. Our findings point to a future of ubiquitous, physically grounded social connection that is not one-size-fits-all but rather individualized to each person’s values.
References
- (1)
- Ambe et al. (2017) Aloha Hufana Ambe, Margot Brereton, Alessandro Soro, and Paul Roe. 2017. Technology individuation: The foibles of augmented everyday objects. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6632–6644.
- Aron et al. (1992) Arthur Aron, Elaine N Aron, and Danny Smollan. 1992. Inclusion of other in the self scale and the structure of interpersonal closeness. Journal of personality and social psychology 63, 4 (1992), 596.
- Bâce et al. (2016) Mihai Bâce, Teemu Leppänen, David Gil De Gomez, and Argenis Ramirez Gomez. 2016. ubiGaze: ubiquitous augmented reality messaging using gaze gestures. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. 1–5.
- Barden et al. (2012) Pollie Barden, Rob Comber, David Green, Daniel Jackson, Cassim Ladha, Tom Bartindale, Nick Bryan-Kinns, Tony Stockman, and Patrick Olivier. 2012. Telematic dinner party: designing for togetherness through play and performance. In Proceedings of the Designing Interactive Systems Conference. 38–47.
- Beckman and Mazmanian ([n.d.]) Christine M. Beckman and Melissa Mazmanian. [n.d.]. Dreams of the Overworked: Living, Working, and Parenting in the Digital Age. Stanford University Press.
- Brewer et al. (2005) Johanna Brewer, Amanda Williams, and Paul Dourish. 2005. Nimio: an ambient awareness device. In Proc. of ECSCW 2005.
- Caine (2016) Kelly Caine. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 981–992. https://doi.org/10.1145/2858036.2858498
- Cao et al. (2010) Xiang Cao, Abigail Sellen, A.J. Bernheim Brush, David Kirk, Darren Edge, and Xianghua Ding. 2010. Understanding Family Communication across Time Zones. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (Savannah, Georgia, USA) (CSCW ’10). Association for Computing Machinery, New York, NY, USA, 155–158. https://doi.org/10.1145/1718918.1718947
- Chai et al. (2017) Min Zhen Chai, Alessandro Soro, Paul Roe, and Margot Brereton. 2017. Cooking together at a distance: Sustain connectedness for long distance families. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2437–2444.
- Chang et al. (2001) Angela Chang, Ben Resner, Brad Koerner, XingChen Wang, and Hiroshi Ishii. 2001. LumiTouch: an emotional communication device. In CHI’01 extended abstracts on Human factors in computing systems. 313–314.
- Charmaz (2006) Kathy Charmaz. 2006. Constructing grounded theory. Sage Publications, London ; Thousand Oaks, Calif.
- Davis-Owusu et al. (2019) Kadian Davis-Owusu, Evans Owusu, Lucio Marcenaro, Carlo Regazzoni, Loe Feijs, and Jun Hu. 2019. Towards a deeper understanding of the behavioural implications of bidirectional activity-based ambient displays in ambient assisted living environments. In Enhanced Living Environments. Springer, 108–151.
- Dey and de Guzman (2006) Anind K Dey and Ed de Guzman. 2006. From awareness to connectedness: the design and deployment of presence displays. In Proceedings of the SIGCHI conference on human factors in computing systems. 899–908.
- Fender and Holz (2022) Andreas Rene Fender and Christian Holz. 2022. Causality-preserving Asynchronous Reality. In CHI Conference on Human Factors in Computing Systems. 1–15.
- Fong et al. (2013) Allan Fong, Zahra Ashktorab, and Jon Froehlich. 2013. Bear-with-me: an embodied prototype to explore tangible two-way exchanges of emotional language. In CHI’13 Extended Abstracts on Human Factors in Computing Systems. 1011–1016.
- Gooch and Watts (2011) Daniel Gooch and Leon Watts. 2011. A design framework for mediated personal relationship devices. In Proceedings of HCI 2011 The 25th BCS Conference on Human Computer Interaction 25. 237–242.
- Grivas (2006) Konstantinos Grivas. 2006. Digital Selves: Devices for intimate communications between homes. Personal and Ubiquitous Computing 10, 2 (2006), 66–76.
- Häkkilä et al. (2018) Jonna Häkkilä, Hong Li, Saara Koskinen, and Ashley Colley. 2018. Connected candles as peripheral emotional user interface. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia. 327–333.
- Hassenzahl et al. (2012) Marc Hassenzahl, Stephanie Heidecker, Kai Eckoldt, Sarah Diefenbach, and Uwe Hillmann. 2012. All you need is love: Current strategies of mediating intimate relationships through technology. ACM Transactions on Computer-Human Interaction (TOCHI) 19, 4 (2012), 1–19.
- Hays and DiMatteo (1987) Ron D Hays and M Robin DiMatteo. 1987. A short-form measure of loneliness. Journal of personality assessment 51, 1 (1987), 69–81.
- Heshmat et al. (2020) Yasamin Heshmat, Carman Neustaedter, Kyle McCaffrey, William Odom, Ron Wakkary, and Zikun Yang. 2020. Familystories: Asynchronous audio storytelling for family members across time zones. In Proceedings of the 2020 chi conference on human factors in computing systems. 1–14.
- Holtzblatt and Beyer (2017) Karen Holtzblatt and Hugh Beyer. 2017. The affinity diagram. Contextual design (2017), 127–146.
- Hutchinson et al. (2003) Hilary Hutchinson, Heiko Hansen, Nicolas Roussel, Björn Eiderbäck, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, and Helen Evans. 2003. Technology Probes: Inspiring Design for and with Families. In Proceedings of the Conference on Human Factors in Computing Systems - CHI ’03. ACM Press, Ft. Lauderdale, Florida, USA, 17. https://doi.org/10.1145/642611.642616
- Ishii and Ullmer (1997) Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’97. ACM Press, Atlanta, Georgia, United States, 234–241. https://doi.org/10.1145/258549.258715
- Ishii et al. (1998) Hiroshi Ishii, Craig Wisneski, Scott Brave, Andrew Dahley, Matt Gorbet, Brygg Ullmer, and Paul Yarin. 1998. ambientROOM: integrating ambient media with architectural space. In CHI 98 conference summary on Human factors in computing systems. 173–174.
- Judge et al. (2011) Tejinder K. Judge, Carman Neustaedter, Steve Harrison, and Andrew Blose. 2011. Family Portals: Connecting Families through a Multifamily Media Space. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems - CHI ’11. ACM Press, Vancouver, BC, Canada, 1205. https://doi.org/10.1145/1978942.1979122
- Judge et al. (2010) Tejinder K. Judge, Carman Neustaedter, and Andrew F. Kurtz. 2010. The Family Window: The Design and Evaluation of a Domestic Media Space. In Proceedings of the 28th International Conference on Human Factors in Computing Systems - CHI ’10. ACM Press, Atlanta, Georgia, USA, 2361. https://doi.org/10.1145/1753326.1753682
- Kim et al. (2015) Jina Kim, Young-Woo Park, and Tek-Jin Nam. 2015. BreathingFrame: An inflatable frame for remote breath signal sharing. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction. 109–112.
- Kim et al. (2017) Kangsoo Kim, Divine Maloney, Gerd Bruder, Jeremy N Bailenson, and Gregory F Welch. 2017. The effects of virtual human’s spatial and behavioral coherence with physical objects on social presence in AR. Computer Animation and Virtual Worlds 28, 3-4 (2017), e1771.
- Langlotz et al. (2013) Tobias Langlotz, Holger Regenbrecht, Stefanie Zollmann, and Dieter Schmalstieg. 2013. Audio stickies: visually-guided spatial audio annotations on a mobile augmented reality platform. In Proceedings of the 25th Australian computer-human interaction conference: augmentation, application, innovation, collaboration. 545–554.
- Lee et al. (2022) Kyungjun Lee, Hong Li, Muhammad Rizky Wellyanto, Yu Jiang Tham, Andrés Monroy-Hernández, Fannie Liu, Brian A. Smith, and Rajan Vaish. 2022. Exploring Immersive Interpersonal Communication via AR. https://doi.org/10.48550/ARXIV.2211.15084
- Li et al. (2018) Hong Li, Jonna Häkkilä, and Kaisa Väänänen. 2018. Review of unconventional user interfaces for emotional communication between long-distance partners. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–10.
- Lin et al. (2011) Chun-Yi Lin, Cheng-Ting Chang, Meng-Tsen Chen, Zoeh Ruan, Alan Hsueh, Yi-Yang Chang, and Ren-Hung Hwang. 2011. Nunote: an augmented reality social note posting service. In International Conference on Ubiquitous Intelligence and Computing. Springer, 94–108.
- Lottridge et al. (2009) Danielle Lottridge, Nicolas Masson, and Wendy Mackay. 2009. Sharing empty moments: design for remote couples. In Proceedings of the SIGCHI conference on Human factors in computing systems. 2329–2338.
- Mankoff et al. (2003) Jennifer Mankoff, Anind K Dey, Gary Hsieh, Julie Kientz, Scott Lederer, and Morgan Ames. 2003. Heuristic evaluation of ambient displays. In Proceedings of the SIGCHI conference on Human factors in computing systems. 169–176.
- Matthews et al. (2003) Tara Matthews, Tye Rattenbury, Scott Carter, Anind Dey, and Jennifer Mankoff. 2003. A peripheral display toolkit. University of California, Berkeley Technotes, UCB//CSD-03-1258 168 (2003).
- McDonald et al. (2019) Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and Inter-Rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 72 (nov 2019), 23 pages. https://doi.org/10.1145/3359174
- Merriam and Tisdell (2015) Sharan B Merriam and Elizabeth J Tisdell. 2015. Qualitative research: A guide to design and implementation. John Wiley & Sons.
- Nassani et al. (2015) Alaeddin Nassani, Huidong Bai, Gun Lee, and Mark Billinghurst. 2015. Tag it! AR annotation using wearable sensors. In SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications. 1–4.
- Nawahdah and Inoue (2013) Mamoun Nawahdah and Tomoo Inoue. 2013. Virtually dining together in time-shifted environment: KIZUNA design. In Proceedings of the 2013 conference on Computer supported cooperative work. 779–788.
- Neustaedter et al. (2015) Carman Neustaedter, Carolyn Pang, Azadeh Forghani, Erick Oduor, Serena Hillman, Tejinder K Judge, Michael Massimi, and Saul Greenberg. 2015. Sharing domestic life through long-term video connections. ACM Transactions on Computer-Human Interaction (TOCHI) 22, 1 (2015), 1–29.
- Pousman and Stasko (2006) Zachary Pousman and John Stasko. 2006. A taxonomy of ambient information systems: four patterns of design. In Proceedings of the working conference on Advanced visual interfaces. 67–74.
- Schmeer and Baffi (2010) Johanna Schmeer and Tom Baffi. 2010. Touch trace mirror: asynchronous, collaborative messaging as a concept for creating a relatedness experience. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction. 303–304.
- Sereno et al. (2020) Mickael Sereno, Xiyao Wang, Lonni Besançon, Michael J Mcguffin, and Tobias Isenberg. 2020. Collaborative work in augmented reality: A survey. IEEE Transactions on Visualization and Computer Graphics (2020).
- Siio et al. (2002) Itiro Siio, Jim Rowan, and Elizabeth Mynatt. 2002. Peek-a-drawer: communication by furniture. In CHI’02 extended abstracts on Human factors in computing systems. 582–583.
- Singhal et al. (2017) Samarth Singhal, Carman Neustaedter, Yee Loong Ooi, Alissa N Antle, and Brendan Matkin. 2017. Flex-n-feel: The design and evaluation of emotive gloves for couples to support touch over distance. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 98–110.
- Surale et al. (2022) Hemant Bhaskar Surale, Yu Jiang Tham, Brian A Smith, and Rajan Vaish. 2022. ARcall: Real-Time AR Communication using Smartphones and Smartglasses. In Augmented Humans 2022. 46–57.
- Tolmie et al. (2002) Peter Tolmie, James Pycock, Tim Diggins, Allan MacLean, and Alain Karsenty. 2002. Unremarkable computing. In Proceedings of the SIGCHI conference on Human factors in computing systems. 399–406.
- Voida et al. (2008) Amy Voida, Stephen Voida, Saul Greenberg, and Helen Ai He. 2008. Asymmetry in Media Spaces. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (San Diego, CA, USA) (CSCW ’08). Association for Computing Machinery, New York, NY, USA, 313–322. https://doi.org/10.1145/1460563.1460615
- Weiser (1991) Mark Weiser. 1991. The Computer for the 21 st Century. Scientific american 265, 3 (1991), 94–105.
- Weiser and Brown (1997) Mark Weiser and John Seely Brown. 1997. The coming age of calm technology. In Beyond calculation. Springer, 75–85.
- Yu and Inakage (2022) Xuqin Yu and Masa Inakage. 2022. Shared Wind: Creating Awareness by Nature-related Ambient Design for Emotional Communication. In Augmented Humans 2022. 334–336.