10gui window management not innovative enough

A friend suggested that I look at the R. Clayton Miller’s 10gui video (2009) for ideas on window management and interaction.

The video makes some interesting observations about human-computer interactions (HCI):

• mice excel at pointing on the screen without obstructing the screen
• multi-touch should be extended to use all digits on the hand, not just 1 or 2 – but all 5.
• both hands can create touch combinations that are interesting ( see 6:42 mark in the video )
• New windows are overlaid on top of old windows in a rather cluttered manner.

Clayton Miller’s proposal involves a medium-size touch surface placed in front of the keyboard. All ten fingers are used to interact with the UI. Different combinations and number of fingers mean different operations.

Clayton Miller’s basic premise that HCI should no longer be confined to 2D interactions is quite correct. However, the proposal does not recognize the full extent of the mouse/keyboard limitations. As a result the proposal is at best an incremental improvement over what Apple offers currently. Furthermore, Clayton’s proposal assumes a desktop computer configuration. Mobile, tablet, and laptop compute configurations are ignored.

Additional limitations that Clayton must bring into the picture and address in order to be truly revolutionary:

• extensive mouse movement causes carpal tunnel syndrome
• mouse/trackpad movement requires a dedicated surface
• mouse is not useful for mobile devices
• mobile devices use a touch screen with the downside that Clayton points out in his video.
• physical handicaps of users:
  • lost digits,
  • diseases that impact muscle control
  • mouse and trackpad are still 2D surfaces and operations

Clayton needs to update this video to consider these technologies:

• Microsoft Kinect‘s motion capture eliminates the need for direct device
  manipulation
• Kinect and the Wii introduced acceleration, 3D motion and movement into the HCI arena.
• Users’ physical limitations
• Eye motion and tracking to make computers more accessible to users.
• Mobile devices in particular field use
• Non-Desktop interactions