Mission statement
General Information

Discover the stories saved in your pocket

Your smart phone collects more data about you than you realize. iYou™ is an iTunes plug-in that visualizes your iPhone’s unexposed data and makes your communication habits transparent.

Did you ever wonder what your data tells you about your relationships with your friends? Who do you wink at the most? Or who do you contact when you feel happy? Combining all the information from different apps into the iYou™ software, allows you to better understand your circles of contacts and the ways you communicate with them.

iYou™ is an open software project resulting from a research project by Niels Schrader and Bert Kommerij, currently being developed in collaboration with Waag Society and software developer Michał Ejdys. The project is committed to openness of data access and the understanding of media transparency.

Download

To download the current version go to the download chapter.

Newsletter

Receive updates about the iYou™ project via email by subscribing to our newsletter.

News

← Previous  Next →

iYou™ at Pixelache Festival 2011, Helsinki

In spring 2011, iYou™ was presented during the Pixelache Festival of Electronic Art and Subcultures. Together with the iYou™ prototype, the research results were exhibited on Suomenlinna island in Galleria Augusta to gather the first extensive feedback by a broader audience.

iYou™ on WeTransfer

To support our mission, feel free to use the new iYou™ Channel on WeTransfer.com for extensive file transfer.
→ iyou.wetransfer.com

Privacy statement

iYou™ takes privacy obligations very seriously as it was initiated with the goal to alert smartphone users on the risks of storing a high amount of personal data in one single medium.

When launched, iYou™ reads through the backup files created by iTunes. It saves the processed data locally to cache relevant information and accumulate data that otherwise would be overwritten by the next backup. If the user wishes to include information form his social networks, permission has to be granted for each platform separately.

The graphs and statistics are created locally on the user’s computer. None of the sensitive information ever leaves the device.

The code of iYou™ will be published open-source shortly to allow users to inspect the code and help improving the software.

Open source

Future plans

iYou™ will be developed as an open source application for Android and iPhone platforms. A long-term objective is to set up a freely available framework that allows users to create various plug-ins. These plug-ins are supposed to supply the system with input (i.e. harvest data from different types of devices) and provide output (i.e. generate statistics or visualizations).

Project description
Concept

Challenge

Mobile phones have long since stopped being mere communication tools. With the introduction and continued development of the smartphone, mobile devices have increasingly developed into universally deployable digital helpers, ubiquitous tools for multi-form communication. In doing so they store an ever-increasing amount of personal data: sms, movies, pictures, call histories, but also emails, social network events and so on.

Data management rarely goes further than grouping and categorizing information in simple categories like time, address, media type. But what if smartphones were smart enough to link data semantically across various media types? What if new stories emerge from our personal data archives? And what if the smartphone could be a tool that helps us filter the never-ending information overload?

Introducing iYou™

iYou™ is a personal, mobile data guide, which confronts the user with a detailed profile derived purely from his media behavior. Conceptualized for iPhone and Android, the application aggregates, clusters and visualizes data the user never knew was collected. It is designed to break the predefined structure of the data buried in various applications and present it non-hierarchically instead.

iYou™ initially began as a research project to make private communication patterns transparent and encourage their understanding. The idea was to create not only statistics based on the data physically stored on the smartphone, but also to allow the user to interpret pieces of information that he unconsciously accumulates day by day.

In their project → iPhone Tracker, Alasdair Allan and Pete Warden revealed that the iPhone was gathering years of data tracking the locations of the GSM and Wi-Fi networks the user was connected to. By doing so, Apple’s iPhone is leaving behind a clear picture of the user’s personal whereabouts and preferred locations. This database on its own is already a valuable source of information. And when combined with other easily available sources, it becomes a profitable tool for business – a tool that should be under control of the user!

It seems Johann Wolfgang von Goethe (1749–1832) had a similar vision when he said “Tell me whom you frequent, and I will tell you who you are.” Even centuries ago, people realized that our relationships define who we are. These days, we don’t even have to say anything, our smartphone will speak for us. If only Goethe had an iPhone…

Media literacy

The goal of the project is to create a personal, mobile data guide, which reflects directly on the media habits of a mobile phone user. It addresses relevant issues of contemporary media:

  • insight into the data stored on a mobile phone (media literacy)
  • mobile communication behavior as part of your identity
  • awareness of how personal data can be exploited by third parties
  • mastering the problem of daily data overload

The project aims to contribute to and elevate the debate on media literacy. The current discussion focuses largely on online behavior and the risks involved. Media literacy, however, concerns the skills and the mentality that people need to consciously, critically and actively participate in the technically mediated world — no matter what medium: mobile, online or otherwise.

Background

The concept for iYou™ was conceived in 2010 during a 3 month → workshop at the Mediafund@Sandberg Masterclass by Niels Schrader and Bert Kommerij in collaboration with Jan van Bruggen. The topic of this workshop was exploring the database as a narrative form.

Research
Approach

iYou™ as a research project investigates the user’s communication behavior and allows to draw his individual communication profile. The source data for the analysis is the backup of the iPhone. A couple of interesting questions emerge from this rich set of information.

Who do I speak to most frequently? Are there any special words that are used only by my group of friends? Do I like to keep in contact with short communications every day or a long communications once every couple of months? Can the user data predict for me who I am best as keeping in touch with or say, remind me of a good friend I used to be in touch with frequently and haven’t spoken to in a while? These kind of questions can be easily answered by analyzing application usage on your phone.

Proof of concept

In the proof of concept, a small subset of all the available information was analyzed. The goal was to explore the relationships between different data sets and understand their value. The developed visualizations revealed communication patterns when analyzing multiple data sets together. This served as a basis to a more detailed research.

Data mining

During the process of data mining, concrete research question were defined to identify trends in the user’s communication behavior. The backup of iPhone user’s data served as a database. Each question was translated into a SQL query to pull out the related records and draw a graph displaying the relevant information.

Research questions

The research questions targeted a full range topics like social preferences (Whom do I contact regularly and how?), personal mood records (Which kind of smiley is related to which context?) and user profiling (What are the links stored on my iPhone?). So far, 9 questions have been defined:

  1. Whom do I contact regularly and how?
    To disclose the user’s favorite communication channels linked to a specific contact.
  2. Which application runs the most frequently?
    To discover the user’s favorite apps and the related media identity.
  3. What are the most frequent keywords I use?
    To capture the topic of the day and determine the character of a relationship with another person.
  4. Which kind of smiley is related to which context?
    To detect the user’s emotions that are linked to different people from his contacts.
  5. How do I communicate throughout the day / week / year?
    To reveal the peaks and valleys of the user’s communication habits.
  6. What are the links stored on my iPhone?
    To the user’s fields of interest and areas of activity.
  7. What are the places I visit?
    To locate the user’s whereabouts.
  8. How quickly do I respond to communication from others?
    To explore the user’s social feedback.
  9. What kind of music have I been listening to?
    To determine what are the user’s favorite bands and genres of the music and establish his taste in music.
Proof of concept

Concept

The first draft of how to approach the large data repositories was sketched as a graph in which content is stored in data structures nested in the individual applications (see figure 1). This representation follows the current practice of navigating on a smartphone. Individual items are clustered around their respective applications and the only way to access them is to go through the corresponding piece of software.

Based on this observation, the project goal was to explore the semantic links between different types of data: looking at the information rather than its default classification.

Proof of concept in a multi-source visualization

To evaluate the concept of connections between the self-contained iPhone applications, a set of 3 different data types was compiled and processed into an SQL database. Subsequently, the data from each source was grouped into separate columns (calls, sms and Facebook messages) and organized along a timeline. Gaps can be interpreted as periods without activity in the given source.

By drawing a link between items of the same person, keyword or time of the day the hierarchical structure is transformed into a linear thread spanning different data sets. Surprisingly, this enables not only an alternative way of traversing the data, but also reveals different communication patterns of the user.

Communication patterns

The threads for the sample user’s contacts show not only which means of communication are used, but also what style the user has (e.g. replying to an sms with a call). Moreover, it indicates if the communication is occasional (see figure 2) or regular over the time (figure 3).

Analyzing the thread for a keyword reveals its favorite media. One can see that location-based words like Amsterdam and home are mainly mentioned in sms, whereas emotional expressions like love and happy are spread over sms and Facebook (see figure 4 to 7).

Finally, inspecting the visualization for the time of the day shows when the given means of communication are used the most. For example, communication can shift from calls and sms during the working hours (see figure 8) to Facebook and sms during leisure time (figure 9).

Research questions
Whom do I contact regularly and how?

Whom do I contact regularly and how?

Probably the most natural question to pose, when analyzing the user’s communication patterns. It strongly relates to the issue of media identity as it reveals not only the most frequently contacted person, but also the favorite communication channels.

Which application runs the most frequently?

Which application runs the most frequently?

Am I an enthusiastic gamer? Or a committed news reader? Or a passionate shopper? Monitoring the usage of software allows to generate a detailed user profile. Knowing your favorite apps, iYou™ is able to determine your favorite communication channels.

What are the most frequent keywords I use?

What are the most frequent keywords I use?

What’s on my mind? What am I talking about? This analysis gives insight into the topic of the day and also reveals different communication styles of the user. This allows not only to determine the character of a relationship with another person, but also to detect the context of the communication in the given language (formal/informal).

Which kind of smiley is related to which context?

Which kind of smiley is related to which context?

Smileys reflect the mood of the given message, and indirectly — the user’s emotional state. The type of these emoticons ranges from happy to sad and from winking to weeping. Plotting the usage of smileys over the time gives the user insight into the happy periods. This could potentially disclose how contagious these emotions are in a dialog.

How do I communicate throughout the day / week / year?

← Previous  Next →

How do I communicate throughout the day / week / year?

This research questions addresses the user’s communication activities (calls, Facebook messages, e-mail) and shows aggregated values for different time intervals. It unveils the user’s communication habits and indicates the peaks and valleys of social activity across a day, a week and a year.

What are the links stored on my iPhone?

What are the links stored on my iPhone?

Every hyperlink on the iPhone is stored for a reason: saved as a bookmark, recommended via Facebook, accumulated in the browser history or assigned to a contact in the address book. Together, they reflect the owner’s frame of reference, as they represent the user’s fields of interest. iYou™ finds all these websites and extracts keywords from their metadata to detect the user’s areas of activities (e.g. architecture, sailing, nature).

What are the places I visit?

What are the places I visit?

iPhone stores a history of GSM and Wi-Fi networks the user was connected to. This information allows to visualize the user’s whereabouts, and depict his personal habits as well as unusual locations.

How quickly do I respond to communication from others?

How quickly do I respond to communication from others?

The user receives an sms or incoming call that he doesn’t answer. After a while, he decides to react (call or text back). iYou™ captures this reaction time to explore the user’s social feedback. This allows to estimate the significance of a call / sms or even the person related to it.

What kind of music have I been listening to?

What kind of music have I been listening to?

iTunes remembers the user’s playlists and some statistics on how they are played. Analyzing this information helps to determine the favorite bands and genres of music. This in terms enables establishing the taste in music and even detecting the user’s mood.

Prototype
Description

iYou™ is prototyped for iPhone in the form of an iTunes plug-in. It runs on your stationary computer and synchronizes with your mobile device every time you connect your iPhone to the computer. The iYou™ plug-in accesses the latest backup created by iTunes in the directory located in ˜/Library/Application Support/MobileSync/Backup/.

Interface

iYou™ is designed as a globe that consists of all the individual communication events stored on your iPhone (e.g. phone calls, Facebook entries and bookmarks). The events are arranged chronologically along the globe’s equator starting with the user’s most recent item in the front and concluding with the oldest event of the back-up after a full turn. For a given day, the events are distributed along the corresponding meridian, starting with morning in the south, through midday on the equator and finishing with midnight in the north.

Related events are linked by threads generated for example by keywords, timestamps, locations, contacts. By zooming in and out as well as navigating in all directions, the user can dive into his data, follow the relationships and discover the stories behind them.

Download

Installation

Download the iYou™ plug-in from here. Unzip the downloaded file and place the bundle into the ˜/Library/iTunes/iTunes Plug-ins/ folder. The software is tested in iTunes 10.5.x on Mac OS X 10.6.8 and 10.7.2. After starting iTunes, go to the View menu and select iYou™ as the Visualizer. Show the Visualizer by pressing command-T.

To access the Library on Max OS X Lion, simply hold down the alt key while accessing the Go menu of the Finder.

Features

The current version iYou™ 0.5 beta includes the following features:

  • analyzing calls and sms from iPhone’s (unencrypted) backup
  • analyzing Facebook messages, if authorized
  • displaying the communication history as a globe
  • visualizing related communication events based on the contact person
  • filtering the items with respect to the type of communication / used emoticons

Mind the current release is a beta version of the prototype! Please excuse any bugs you might encounter and don’t hesitate to drop us a line if you have any feedback you’d like to share.

For the complete release history, please have a look here.

Public appearance
Lectures, exhibitions, articles

Presentations

As part of the project the research has been presented at a number of conferences and lectures, including the following events:

27 March 2012
Carnegie Mellon University (School of Design), Pittsburgh
→ design.cmu.edu

26 November 2011, What You Can Learn From Hidden Mobile Phone Data
Quantified Self Conference, Amsterdam
→ www.quantifiedself.com

16 November 2011, Emzin seminar on visual communication
City Museum of Ljubljana
→ www.icsid.org

6 October 2011, What. How. Why.
AGI Congress, Barcelona
→ www.agicongress.org

4 July 2011, Zukunft der Schrift
University of Applied Sciences, Düsseldorf
→ www.fhd-tm.de

21 April 2011, AGI ‘Home & Abroad’
Stedelijk Museum, Amsterdam
→ www.agi-nl.nl

14 April 2011, Data-driven Exploration of Dynamical Networks
IMéRA, Marseille
→ www.imera.fr

7 April 2011, Zefir7 / Optical interventions
Theater Zeebelt, The Hague
→ www.zefir7.nl

31 March 2011, Data als ‘beautiful evidence’
Waag Society, Amsterdam
→ events.waag.org

12 March 2011, Pixelache — Festival of Electronic Art and Subcultures
Camp Pixelache, Helsinki
→ www.pixelache.ac

12 May 2010, Follow the Money
De Balie, Amsterdam
→ followthemoney.nu

Media appearances

January 2012, One is a fact, two is a story (Page Magazine)
→ www.page-online.de

23 January 2012, iYou bèta-testers gevraagd
→ www.onemorething.nl

16 January 2012, Test de bèta-versie van iYou
→ blog.waag.org

3 December 2011, Je eigen zwarte doos (Vrij Nederland)
→ www.larsanderson.org

26 November 2011, iYou Accumulates Smart Phone Data
→ www.quantifiedself.com

26 August 2011, iYou: Revealing the iPhone’s Hidden Usage Data Logs
→ www.infosthetics.com

Press release

This page is currently under construction. Please come back soon.

Contact
Address

If you are interested in contacting us, please drop us a line via the following 3c6120687265663d226d61696c746f3a6e69656c73406d696e6464657369676e2e696e666f223e652d6d61696c3c2f613e. Questions or comments are always welcome.

Mind Design
Postjesweg 1
Het Sieraad, Studio K34-K36
NL 1057 DT Amsterdam
→ www.minddesign.info

Waag Society
Piet Heinkade 181 A
Pakhuis de Zwijger
NL 1019 HC Amsterdam
→ www.waag.nl

Credits

iYou™ is an open software project resulting from a research project by Niels Schrader and Bert Kommerij. Further development and implementation is now accomplished in cooperation with Waag Society and software developer Michał Ejdys. The project is financially supported by the → Dutch Cultural Media Fund.

Sincere thanks go to former collaborator Jan van Bruggen for supporting the iYou™ research, Pascal de Man for additional programming and Diane Peters for copy-editing.

Biographies

Niels Schrader: graphic designer, founder of Mind Design, lecturer at the Royal Academy of Art in The Hague, member of AGI — Alliance Graphique Internationale and initiator of the iYou™ project.

Bert Kommerij: scriptwriter, director, award winning radio / filmmaker at the Dutch broadcasting station NTR and initiator of the iYou™ project.

Jan van Bruggen: freelance software developer, specialized in unconventional online applications.

Dick van Dijk: concept developer at Waag Society, head of the Urban Reality Lab, background in Business Economics and History of Art.

Michał Ejdys: software developer specialized in distributed systems, cooperating with Mind Design.

Edwin van Ouwerkerk Moria: back end developer at Waag Society, specialized in mobile apps and open data challenges.

Laurens Schuurkamp: front end developer at Waag Society, specialized in 3D, motion and interaction design, background in architecture.

Newsletter

Receive updates about the iYou™ project via email by subscribing to our newsletter.

First name:
Family name:
Email address:
Captcha phrase: Refresh