An Apple patent (number 20110246891) has appeared at the US Patent & Trademark Office for pushing a graphical user interface to a remote device with display rules provided by the remote device. In other words, Apple is working on ways to beef up the remote control abilities of iOS devices — and incorporate new abilities along these lines.
The invention relates generally to accessories for portable media devices and in particular to allowing a portable media device to provide a user interface for controlling the media device to an accessory. Per the patent, a graphical user interface (“GUI”) can be presented on a remote control accessory device that has user input and display devices. The portable media device can provide the accessory with a GUI image to be displayed. The accessory can specify the configuration of the remote GUI image and send that information to the portable media device.
The portable media device can generate a GUI image based on the configuration and send the GUI image to the accessory. The accessory can display the GUI image on one part of its display and other information on another part of its display. The portable media device can define different GUI image types for use with different types of input controls of the accessory. Based on which input control is currently active, the portable media device can send the corresponding GUI image to the accessory for display on the accessory display. The inventors are Emily Schubert, Shyam Toprani, Thomas M. Alsina and Szu-Wen Huang.
Here’s Apple’s background and summary of the invention: “Portable media devices allow users to store media content (music, videos, photos, audiobooks, etc.) and access stored media content from nearly anywhere. Some portable media devices also allow users to connect other devices (referred to herein as ‘accessories’ or ‘accessory devices’) to the portable media device, thereby enhancing some aspect of the portable media device’s operation. For example, most portable media devices provide a headphone jack for listening to audio content. Some accessories provide speakers, thus allowing the user to share the sound with others, or to listen in environments where use of headphones might not be feasible (such as while driving).
“In some instances, an accessory can be used to control operation of a portable media device; such accessories are referred to herein as ‘remote control devices.’ The remote control device can send commands to the portable media device indicating a user-requested operation (such as starting or pausing playback, skipping to the next track, returning to a previous track, fast-forward or rewind within a track, etc.).
“The portable media device can execute the requested operation, thus allowing the user to operate the portable media device without touching the portable media device. Such remote control operation can be particularly useful in situations where it is inconvenient for a user to manipulate the portable media device’s interface directly. For example, portable media devices tend to be small, with relatively small controls and display screens. Thus, it can be difficult for a user to operate such a device while driving or from across a room.
“Some remote control devices provide a graphical user interface (GUI) and allow the user to perform more advanced functions such as browsing a database of stored content, selecting content to play, etc. For instance, Johnson Controls International has been developing a ‘Mobile Device Gateway’ for use in vehicles; the system includes a connection point for a portable media device and a console that provides an audio/visual interface. But existing remote GUIs are defined and controlled by the remote control device, and consequently, they may bear little resemblance to a GUI supplied by the portable media device itself.
“Certain functions available on the portable media device (such as browsing or searching a database, adjusting playback settings, etc.) may be unavailable or difficult to find. Thus, a user may not be able to perform desired functions. Further, GUIs provided for the same portable media device by different remote control devices might be quite different, and the user who connects a portable media device to different accessories with remote control may find the inconsistencies frustrating. It would, therefore, be desirable to provide a more consistent remote user interface experience.
“Embodiments of the present invention relate to providing a graphical user interfaceon a remote control accessory device, where the GUI can be defined and managed by a portable media device rather than the accessory device. The accessory device can provide a combination of user input and visual feedback devices, such as a video screen for presenting information and feedback to a user, along with buttons, knobs, touchscreen and/or touchpad for receiving user input.
“The portable media device can provide the accessory with an image (referred to herein as a ‘remote’ GUI image) to be displayed on the video screen; the image can include various user interface elements that can resemble or replicate a “native” GUI provided directly on the portable media device. The accessory can send information to the portable media device indicative of a user action taken in response to the displayed image; such information can indicate, for example, that a particular button was pressed or that a particular portion of a touch-sensitive display screen was touched by the user. The portable media device can process this input to identify the action requested by the user and take the appropriate action.
“The action may include providing to the accessory an updated GUI image to be displayed, where the updated GUI image reflects the user action. In some embodiments, the accessory can provide access to various third party applications resident of the portable media device by interacting with the display of the accessory. In some embodiments, where the display of the accessory is touch-sensitive, the user can directly or indirectly touch a portion of the screen associated with an identifier for a third party application in order to access the corresponding third party application.
” In some embodiments, a portable media device can supports a number of different functionalities (e.g., audio playback, video playback, still image playback, telephone, World Wide Web browser, maps and navigation, etc.), and an accessory can use the remote GUI to access any or all of these functionalities. The set of functionalities accessible via the remote GUI can be determined by the portable media device or the accessory.
“In one embodiment, the accessory can specify which of the functionalities supported by the portable media device should be made available via the remote GUI. In another embodiment, the portable media device can make the selection. In either case, the selection can be based in part on information about the operating environment or status of the accessory.”
— Dennis Sellers