Computer Science

Share this article:

Computer Science

  • Join our comunity:

Smartphone market stagnating? Time to break the glass and seek disruptive solutions

By: , Posted on: May 5, 2016

With the news that Apple iPhone sales have fallen recently and other manufacturers have similarly experienced a reduction in consumer appetites, Matt Jones and Simon Robinson (authors of “There’s Not an App for That”) consider potential disruptive screen technologies that could reinvigorate the market. They will be presenting work at the forthcoming ACM CHI conference in San Jose that they carried out with their colleagues Jen Pearson and Matheus Fernandes Torquato in Swansea, and Celine Coutrix, Juan Rosso and Laurence Nigay in Grenoble, France.

While entrancing, the flat, dark, dead screens of today’s mobiles seem in need of enlivening. Like the pool that Narcissus stared into (see the book’s cover) it is seductive but ultimately, potentially, a distraction from the tangible, shape-ful, textured world around us.

Over the past couple of years, along with researchers in Grenoble, France, we’ve been thinking about how hard, flat mobile screens might one day morph to provide hands-in and hands-on interaction. Commercially, there is already a taste of what could come, as Tactus technologies have shown. Take a look at some of the more radical ideas we’ve considered in the Figure below.


Figure 1.
Example (de)form factors and interaction possibilities. (a) A slider with a dynamic, variable size button emerges from the screen; (b) A door handle element can be grabbed and turned; the handle’s resistance can be varied to communicate data properties (e.g., the handle might be stiffer if this data vault hasn’t been opened in some time); (c) The deeper the user pushes into the paint pots, the darker the colour they paint with; and, (d) A butterfly flits its wings from the display surface and a user can feel textures as she touches the wings.

And consider this scenario,

Rosie is watching a BBC nature programme on her very large screen HD television set. In her hands she holds a tablet, which can display companion content for the programme. While the large screen shows a butterfly dancing around a garden, her tablet screen deforms to provide a sensation of the flapping wings, indicating that there is content on the second screen. However, she is immersed with the main screen display, and briskly pushes the deformation away to the right of the mobile. The force of her flick is used by the system to gauge her interruptibility as additional content becomes available. See Figure 1(d).

In the paper, “Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction”, that will be presented at CHI, we present the first steps towards some aspects of this vision. The work involved building two prototypes that allowed for continuous control of a widget – think moving a slider or turning a dial – while keeping your eyes off the device and on the surroundings. The video, below, illustrates a set of such tasks that might be useful for a user at home controlling their television or lighting.

The paper details the design of the prototypes and a set of laboratory experiments to evaluate how the systems compare with today’s standard flat screens for the same tasks. As the paper explains, the results are promising, with some significant performance gains and improvements in the more subjective elements of user experience.

We’re continuing the work, having recently gained funding from the UK’s science funder (EPSRC), expanding the team to include a group led by Sri Subramanian in Sussex, researchers at the BBC R&D Centre, and a bioscientist in Swansea who is provoking us to think about how animals deform and reshape.

Our effort is one of many going on in labs around the world to break the glass of today’s conventional thinking (our paper gives pointers to the work we’ve been inspired by). If together we can find the right materials and design the equivalents of “pinch to zoom” or “slide to scroll” on screens that can present feelable textures, shapes and movement, then perhaps even Narcissus would look up and take notice.

There's not an app for that

There’s Not an App for That will make your work stand out from the crowd. It walks you through mobile experiences, and teaches you to evaluate current UX approaches, enabling you to think outside of the screen and beyond the conventional. You’ll review diverse aspects of mobile UX: the screens, the experience, how apps are used, and why they’re used. You’ll find special sections on “challenging your approach”, as well as a series of questions you can use to critique and evaluate your own designs. Whether the authors are discussing real-world products in conjunction with suggested improvements, showcasing how existing technologies can be put together in unconventional ways, or even evaluating “far out” mobile experiences of the future, you’ll find plenty of practical pointers and action items to help you in your day-to-day work.

Use discount code “STC215” at checkout and save 30% on your very own copy!


About the author

Matt Jones is a professor and Head of Department of Computer Science, Swansea University. His research work focuses on human-centered computing with particular emphasis on mobile and ubiquitous computing and resource-constrained communities in regions such as India and South Africa. His work in these contexts has been recognized by an IBM Faculty Award and, from 2014, by a Royal Society Wolfson Research Merit Award. Matt has had many active collaborations and interactions with industry, NGO and Governmental stakeholders including Microsoft Research, Nokia Research and IBM Research. In his spare time he tries to live life face-on with his energetic family, and enjoys nothing more than an exhilarating early morning cycle ride to the glorious beaches of the Gower.

Connect with us on social media and stay up to date on new articles

Computer Science

Computing functionality is ubiquitous. Today this logic is built into almost any machine you can think of, from home electronics and appliances to motor vehicles, and it governs the infrastructures we depend on daily — telecommunication, public utilities, transportation. Maintaining it all and driving it forward are professionals and researchers in computer science, across disciplines including:

  • Computer Architecture and Computer Organization and Design
  • Data Management, Big Data, Data Warehousing, Data Mining, and Business Intelligence (BI)
  • Human Computer Interaction (HCI), User Experience (UX), User Interface (UI), Interaction Design and Usability
  • Artificial intelligence (AI)
Morgan Kaufmann companion resources can be found here You can also access companion materials and instructor’s resources for all our new books on the Elsevier Store. Search by author, title or ISBN, then look for the “Resources” tab on any book page. Looking for companion materials or instructor’s resources for these titles? Connect below: