New ways your smartwatch (and phone) may be spying on you. How worried should you be?

A computer science Masters student Tony Beltramelli at the IT University of Copenhagen has demonstrated that software running on a smartwatch could be used to record a user’s passwords and PINs. He managed this by using the smartwatch’s motion sensors and analysing the patterns of data from the sensors when tapping a keypad to enter a PIN.

Software detecting PIN being entered

Although it is assumed by Beltramelli and others that the application doing the spying would be installed without the user knowing, it is quite possible that a seemingly legitimate app installed from the app store could be doing the spying. This is because access to the sensors is not seen as a security, or privacy risk. Data from the motion sensors is used for controlling aspects of the user interface and so it would be unreasonable to ask a user’s permission to access that data.

How does it work?

Smartwatches like the Apple Watch have two sensors that measure motion; a gyroscope and an accelerometer. Gyroscopes measure the speed and angle of rotation of the watch along three different axes i.e. how fast the device is spinning in any of three directions. Accelerometers measure the acceleration of the device along the same axes. Using these sensors together, apps on the watch can detect specific movement, like for example, lifting the watch to look at the face, which on most smartwatches will cause the watch screen to switch on.

Apple themselves use the data from these sensors to detect when wearers are sitting, standing or moving but stop short of providing any more detail than that.

Beltramelli took data from both sensors on a smartwatch and then applied a type of machine learning to teach his software to detect when specific buttons on a numeric pad were being pressed. This required the software to be “trained” during the learning process, to recognise specific movements of the wearer. However, even without the training, the software was reasonably accurate at identifying the buttons being pressed.

Other approaches

This is not the first time that someone has used motion sensors in a mobile device to carry out keylogging. Other researchers have done similar things on smartwatches and mobile phones. In the case of mobile phones, the sensors can be used to pick up vibrations from a keyboard when the phone is placed on the same surface nearby. Motion sensors can also be used to capture what a user taps onto a mobile phone screen.

How plausible is this attack?

There are a number of limitations that make this type of approach using a smartwatch impractical as an attack against specific targets. For a start, it only works if the person is using the the arm that the watch is on. This may not happen that often as people will tend to use their dominant hand to enter PIN numbers and will wear their watch on their non-dominant wrist. The other problem is that it is one thing to recognise slow deliberate movements as used by Beltramelli in his research. It is another when trying to decipher the more noisy, but probably more common ways in which people enter their PIN on a keypad. There is also the more obvious problem that a PIN is not terribly useful without the information relating to what it is being used for. In the case of a bank card, the PIN is also unusable without the actual physical card.

What is more concerning however is the sophistication by which software and sensors associated with watches and mobile phones can infer what their wearers are doing at any point in time. Motion sensor data, coupled with data from other sensors that measure heart rate could be used to detect a range of very specific activities with the user being unaware. As a matter of privacy, the amount of information that could be inferred by almost any app developer is potentially enormous. This could range from detecting when someone is working and conversely, not working, to wearers sleeping, or even engaging in more “intimate activities”.

PVD+ demonstrating smartwatch controls (Copyright Reuters)

In the meantime however, a Taiwanese company PVD+ has created a more entertaining use for motion sensors on the Apple Watch. PVD+’s software allows an Apple Watch wearer to control the flight of a drone using gestures that are similar to how the Jedi uses the Force to move objects in Star Wars.