It has been awhile since posting to mercurious. However, we’ve been hard at work updating our academic site. This site now documents recent student work:

• Freshmen Processing work
• Muzzi, the digital pet —music player hybrid
• Info Wraps, a mobile informatics research project

It also now features a commercial work samples page that highlights an eight-year career in the interactive media industry. Perhaps the most interesting additions include ancient works from the vault, some almost a decade old.

In fact, getting this site together in this manner has been long overdue.

» New and improved professorial site on parsons.edu

On Teaching Mobile Application Design at Parsons

Dr. Dobb’s Journal, legendary software design publication, published our phone interview on its web portal yesterday on the subject of teaching mobile media design at Parsons Communication Design & Technology. John Dorsey, the editor of Dr. Dobb’s, offered probing questions that got at the nature of the program, the challenges of teaching Flash for application design, and how various tools and platforms can fit together in a technology curriculum at the service of the arts. This is all very interesting when you consider Dr. Dobb’s target audience, clearly hard-core coders and application developers. It’s a good sign when engineer-types are starting to take what’s going on in art and design schools more seriously. These distinctions are very much starting to blur.

» Interview

Hyperbole Essay Muses a Protestant Split Over Jesus Phone

Release a breakthrough product, ready the defenses. Slash prices within 60 days of its release. Pre-test an elaborate consumer store credit program. Maintain control of user experience, revenue streams, and product integrity. Expedite firmware drops to shore up security against an unpredictably vigorous, yet independent, and ingenious rogue software design movement. Monitor ensuing online chatter and surveil the field, ready to deploy assets.

Such is the life of Applists, at the brink of factioning off into a ruthless, fringe element, no longer allegiant to Pope Jobs. These followers protest, disillusioned with the rejection of their effusive prayers and deeds in the name of the product’s church. Emblems of the faith, splendid exemplars of user interface mantras led by installers as elegant as “undocumented features,” spreading customizations and innovations of every whim as if Gnostic Gospel. Each passionate shortcoming of the breakthrough product, much hyped, yet still maligned beyond nitpick, had been redeemed, by painstaking devoted minds, dedicated to unlocking the mysteries of the revolutionary creation, without manual, in disobedience to advice and guidance.

Today the mark is 1.1.1 vs. 1.0.2 — those who have crossed over towards Applist orthodoxy and those resistant, not yet yielding, making due without holy sanctions, verified creed. Perhaps it’s the attraction of a more personal relationship with the Godhead, the sacred code, Kabalah’s sublime intertextual numerology, the open principles of touch-based human interface guidelines and their applications, Gnostic texts canonized by Memex. This divine intimacy of the populace is too much for the orthodoxy to stomach, unmediated transgressors. Only deep within the Vaticanal campus of Cupertino, could the details of iPhone be realized and celebrated through masses. Or could they? The orthodoxy obediently applies edicts from axis mundi, replete with sharia, releasing newly inscribed sacraments of commerce, and trivial revisions to interface ritual, all as sacrifices to the Godhead in the name of stablity, security, and eschatology. The reformists resist the superficialities of bug fixes, feature tweaks, and new revenue streams in favor of purist pursuits of homebrewed exchange and unmediated gadgeteering.

Watch the iPhoneDev splinter group, the prophets Erica Sadun, and her followers, all Digging, the global audience, Natetrue and Nervegas and Nightwatch, all the great disciples of the tripartite theory of Jobs, Ives, and Schiller; now splintering off into a Protestant reformation, rejecting the hegemonies, the hierarchies, the corporate religiosity that puts shareholder before stakeholder, priest before parishoner, mission before congregation, partner before customer, policy before press, business before pleasure.

Other churches reap followers by addressing the misgivings of these loud voices, the difficult choice of switching allegiances, and selecting new identities could be the most drastic reaction to the Applist inquisition. Once dedicated followers, so-called fanboys, mavens, influencers, the variously labeled, occasionally ridiculed, persecuted acolytes of American design purity, objects painstakingly assembled by Chinese workers, clothed in their consumer best, they are lovingly delivered to lavishly spoiled consumers, whose lives change instantly, noticeably, invisibly, despite clear markings and insignia.

Needless to say, we aren’t the only ones jolted into a state of doubt, of second-guessing, a crisis of faith. Our iPhones remain 1.0.2, our AppTap apps continue to dance upon our SummerBoard with a flick, semi-useless, yet somehow indispensable reminders of cherishing the essentials of open, of freedom, of flexibility. We remember not to take for granted the simple joys of the original Applist congregation, enabled to seek more personal relationships with what the breakthrough product enables, beyond expansion and customization: retrogaming, instant messaging, operating systemics, electronic literature, voice recordings, recreation, media redistribution, virtualization, and location-based positioning.

The factionalization inevitably reflects the disconnect between the hierarchy and its Diocese. Cupertino must enforce and enrich its control and influence over the essential elements of its worldview, the device software. There are specific fiduciary duties towards contractual partners and shareholders that assess profound regard in refocusing the consumers attention towards revenue streams and away from endeavors that distract from ongoing tithing. Yet the followers, in their humanistic zeal for coming to know the the device software more intimately, more greatly, more intrinsically, unlocked an interconnection that may not have even been intended for mere mortals. This Pandora’s box exemplifies the detachment of the Tripartite from its congregation. Such a clearly precedented trait of Applists was forgotten, and now fracturing ensues.

Will devotees of Apple fracture into subservient and subversive sects, now prompted to select allegiances?

Specula-palooza

Risking iPhone coverage overexposure, today we ponder one of the most interesting questions about the future of Flash, iPhone and web standards. Despite assurances by Uncle Walt that Apple and Adobe are hard at work on a Flash Player for iPhone, plenty of naysayers, skeptics, and player-haters have voiced strong speculations that Flash will never appear on the iPhone for strategic, practical and technical reasons. A quick scan of comments on various iPhone related entries across the web reveals an almost universal plea amongst everyday users indicating that a dearly missed feature from Mobile Safari is the presence of a mainstream multimedia plugin. In fact, the world’s most popular piece of software in history, is well known to be absent from iPhone.¹

Mobilizing the Means of Production

Those who have written about why iPhone should not have a Flash Player don’t mask their agendas. These voices are usually programmers and developers who have always been hostile to Flash, mostly because it threatens their grip on the means of production, by bringing software and interface design to the masses. Indeed, we’ve seen the worst and the best of the web, as a result. Furthermore, because Flash has always been a mainstream, populist, albeit proprietary media format, it has been deployed for reach and ease of creation, rather than robust performance. When imagining a Flash Player for iPhone, its high-octane thirst for processor cycles does not bode well for battery life.

Monopolizing Web Standards

The second significant argument against iPhone Flash is Apple’s strategy to deploy its WebKit “web standards” platform for all third-party application development. Indeed, the recently redesigned Apple.com site has reduced its use of Flash significantly, in favor of JavaScript and browser based features, so-called AJAX. The argument goes that Apple doesn’t want to forsake its influence on the consumer-level interface design market by inserting Adobe’s trojan horse into the battlefield.² This perspective does make a lot of sense, but leaves out a tremendous amount of nuance, that we’ll investigate here.

A Muted Voice

The assumption that web standards based technology, such as JavaScript, can wholly replace Flash functionality is only somewhat true, especially on iPhone.³ Certainly, the support of JavaScript and its embrace by the web developer community over the past three to five years has changed the face of the web, earning the popular title of version 2.0. However, Flash offers some essential multimedia capabilities that JavaScript alone cannot yet offer. This includes audio support. It’s no coincidence that Bejeweled for iPhone is mute. Especially considering that many iPhone users may have stereo headsets plugged in during use, there are unimaginable uses for audio based applications, especially when combined with locative media technologies. Imagine a sort of audiopedia. The differences don’t start and end with audio support, however. Even Apple’s newly touted Web Gallery feature, part of a .Mac subscription with the iLife personal media suite needs to use Flash for its carousel photo browsing interface.⁴ Indeed, only the Flash Player offers the multimedia engine to manipulate the images as a responsive interface with the reach required on this consumer grade product.

I Want My MTV

Certainly, the most common deployment of Flash Player on the web recently is for web video. It was almost shocking to watch the FLV video format surpass RealPlayer, Windows Media and QuickTime as the most important, influential web video format, in what seemed like a matter of months, with much thanks to YouTube and others. A recent Adobe Flash Player 9 public beta featured H.264 video support, which seems part of its strategy to preserve the dominance of Flash video, especially as Apple and Google migrate towards this non-Flash based video standard. However, until the myriad of embedded SWF FLV players on perhaps billions of web pages get updated to auto-detect the client and deliver the appropriate video by codec, the web will still appear to be littered with missing plugins on Mobile Safari.

Assumptions and Speculations

Let’s proceed with the assumption that Walt Mossberg was correct, and indeed, Apple and Adobe have reached an agreement to release Flash Player iPhone in some manifestation, at some time. Of course, he could be blindly speculating like the rest of us, just running on the fact that it feels crazy for Flash not to be there. But, let’s hope he’s as well-connected and respected as they say he is.⁵ There are several scenarios for the future of Flash iPhone, which should only contribute to the over-saturated discourse by further complicating the biased opinions with an understanding of Adobe’s perspective, previously, and conveniently left out of the discussion.

A Straight Port of Flash Player 9

In this scenario, Adobe compiles the current version of the desktop Flash Player 9 for the ARM processor of iPhone. This scenario would allow all of the existing web-based Flash content to function within Mobile Safari. Authors who create content in ActionScript 3 would enjoy a noticeable improvement of performance and energy efficiency on the iPhone, since this type of content would play in the more recent Flash Virtual Machine, a marked improvement over previous runtime environments, across Windows, Mac OS X and Linux. However, the vast majority of Flash content on the web right now was created in ActionScript 1.0 and 2.0, and so does not take advantage of the improvements to the runtime. Indeed, the skeptics are correct when they assume that running web pages with Flash content, even animated banners, would put an unfortunate drain on the battery.However, what’s crucially missing from a simple straight-on port of Flash Player 9 for iPhone is a substantial confrontation with the multi-touch interface. This is likely the deal-breaker for this scenario. Although, ActionScript and Flash button symbols might offer some means of developing and designing for the iPhone’s multi-touch interface, it’s more than likely that despite the best efforts of Flash coders, the existing means to respond to mouse-based interactions will fall short of the requirements needed to respond to multi-touch. Gestures like pinch, tap-zoom, and flick are difficult to imagine as ActionScript events. Indeed, Adobe probably needs to reckon with the reality that ActionScript needs a true multi-touch API. This could be one of the many reasons why we can only assume that iPhone Flash is in development now, and may be for quite some time, especially based on Apple’s delay in releasing an official iPhone SDK.

Probability: Very Low

Flash Player 10

Looking into 2008, perhaps third or fourth quarter, we have to assume that Adobe will continue to release improved versions of the Flash Player. Version 10 is likely to provide Adobe with the required release cycle needed to fully contend with Apple’s native iPhone application API and SDK release schedule. It’s much more likely that ActionScript libraries will be written that allow true response to the multi-touch gestures, such as pinching, flicking and zoom-tapping. It’s really difficult to imagine a Flash Player on iPhone without this crucial ActionScript API.Furthermore, this allows Adobe to potentially release new versions of the Flash and Flex authoring tools that will compile in Flash Player 10, and subsequently, support a runtime environment that is tuned to the needs of the ARM architecture and precious battery life. It’s been frustrating to read the Flash iPhone haters and their blatant neglect for Adobe’s expertise in the area of building a Flash Player for mobile devices. There has simply been no mention of the possibility of Flash Lite for iPhone.

Probability: Medium

Flash Lite 2.1

Flash Lite is a very special version of the Flash Player for mobile devices, such as Symbian Series 60, Windows Mobile and others. It is not the Flash Player that typically sits in an embedded browser, like the vast majority of Flash content out there. Instead, Flash Lite content exists as standalone, full-screen, mobile applications, or more appropriate mobile media implementations such as a standby screens, wallpapers, screen-savers, and even the device’s native UI.Not only is Flash Lite compiled to the particular device, and so is limited by its processing and memory capabilities, but it also has proven to be very energy efficient, accordingly. Indeed, authors must specifically design and code for Flash Lite. It is in no way, a conversion from desktop Flash content. In this manner, designers and developers alike, are rightly forced to contend with the requirements of mobile media, in terms of interface, content and use-case considerations.It’s possible that Adobe could port Flash Lite to iPhone instead of the expected desktop Flash Player. In this regard, Flash Player would exists as a widget on the SpringBoard home-screen of iPhone, and not as a Mobile Safari plugin. Although it would not fix countless broken plugins in pages that use Flash Player, it would offer mobile media design opportunities for Flash on iPhone. Specifically, Flash Lite already offers APIs to interact with mobile phone specific features, such as triggering the vibrate, detecting battery life, and cell network signal strength. These capabilities are not offered on the desktop version of Flash Player, and may or may not be available in Apple’s official native iPhone application API or SDK. Only time will tell.In this scenario, Flash Lite becomes an avenue for designers and developers familiar with Adobe’s toolsets to create applications that exist as standalone native experiences, rather than embedded modules of Mobile Safari. In many ways, this supports the special needs of mobile media more appropriately than simply making the familiar Flash just work. Imagining a port of Flash Lite for iPhone could mean the opening of a vast market for native iPhone widget applications, designed by designers, and not restricted to hard-core Objective C programmers. Indeed, this is a threatening prospect for those that are eager to carve out a niche in iPhone application development. However, Flash Lite 2.1 probably does not offer the means to react to multi-touch gestures like pinch, flick and tap-zoom, and so it’s probably not the version we will eventually see.

Probability: Low

Flash Lite 3

Just as the desktop Flash Player is constantly updated, the mobile Flash Lite player is also expected to be evolving. It is said, that a forthcoming version of the Flash Lite player, perhaps 3, will bridge the gap between the embedded browser Flash content and the standalone mobile specific Flash Lite content.⁶ In other words, Flash Lite 3 could play not just in the mobile device’s web browser, but could also run within its native operating system environment. This jives with Adobe’s efforts to seed the use of Flash outside of the browser and distribute a desktop based native runtime, called AIR, the Adobe Integrated Runtime. In addition, a Flash Lite 3 would address the concerns of a multi-touch API, and the required energy efficiencies for battery life, memory usage and processor cycling, as well as provide ActionScript to trigger mobile device specific features intrinsic to Flash Lite.In this case, Flash would exist in two manifestations on iPhone: as standalone native applications on the SpringBoard home screen, and as a Mobile Safari plugin, playing the usual desktop based content. This gives developers, designers and end-users the best of all worlds. Flash content can be created with mobile use in mind, considering the unique user interface and energy efficiency required. Everyday users will enjoy not only the full multimedia web they’ve grown accustomed to on the desktop, but also will enjoy a market of native mobile applications that arguably, the Objective C programmers of the world, simply cannot singlehandedly service.

Probability: Medium-High

Conclusions

Flash will appear on iPhone eventually. There is no doubt that Adobe will roadmap this device into its strategy for Flash Player, Flash Lite, or both.Yes, there are significant performance, interface and user considerations that must be addressed as Flash appears on iPhone. Adobe has already demonstrated an accomplished ability to service the mobile media market. It’s only a matter of when, and what form iPhone Flash takes as it appears beneath our beloved glass multi-touch screens.One can only imagine the pressure Adobe’s product planners are putting on its Flash Player team to fit iPhone into the needs of the short-term and long-term strategy. In the short term, Adobe needs to get Flash on iPhone within the next three to six months to capture the required developer and designer audience, and compete with the ascendancy of Mobile Safari and native iPhone applications. In the long term, Adobe needs to get it right, and release a iPhone Flash Player that addresses the specific needs of both mobile media and the vast legacy of desktop Flash content out there. A premature release could spell long term disaster for Flash, as it needs to compete with the rapidly expanding Open Source and Web Standards movement. We haven’t mentioned Microsoft’s entry into the web media space with its recently launched Silverlight platform, but until we stumble upon a site that actually uses it, it’s irrelevant.Although there are many who would like Flash to just go away, because it’s not open source, not free, and tends to be used to bombard us with annoying banner ads and horrible interface design models, Flash is not going away anytime soon. However, how Apple and Adobe navigate the uncharted territory of merging mobile and desktop user experiences along with multi-touch interfaces, will certainly determine the relevance of Flash in the years to come.

Footnotes

1. See Adobe statistics on Flash Player downloads.
2. See Roughly Drafted’s analysis.
3. See A List Apart’s analysis.
4. See Cabel Sasser’s Twitter which claims dibs on this observation.
5. See the profile of Walt Mossberg in the New Yorker.
6. Speculative, but based on reliable, but undisclosed interactions with Adobe. Also, see Bill Perry’s entry on the subject.

Read Packing Lite: A mobile media interface design primer on Adobe’s Developer Center to get my perspective how to adapt to directional-pad based devices in Flash Lite. I detail the process of transferring SWF files to Nokia Series 60 devices and analyze a 4-way menu example from Adobe’s Content Developer Kit (CDK).

» Packing Lite on Adobe.com

Article to be published on Adobe Developer Center

Packing Lite: Getting Started Designing Interfaces for Mobile Media

I wrote an article/tutorial for Adobe’s Developer Center to be published at the end of August covering how to get started designing interfaces in Flash Lite on Nokia Series 60 devices. It details how to get equipped for mobile media design using Flash and the Flash Lite platform, install Flash Lite content on Nokia S60 phones with Bluetooth or Nokia’s PC Suite, discusses the unique interface design challenges, and looks at an example, highlighting the issues.

Stay tuned, as I’ll be sure to post the link to the article when it goes live. Article pushed live at 5 PM PST, August 17, 2007←

A collaboration between the Academy of Arts and Design, Tsinghua University, Beijing and Communication Design and Technology, Parsons The New School for Design

Objective

Develop a design prototype to present to the Art and Design Academy and BOCOG proposing an interactive Mobile Visitors Guide for the 2008 Beijing Olympiad. Comprising of a desktop web product and a downloadable mobile media component, the Mobile Visitors Guide will offer travelers to Beijing helpful information about Olympic events, venues, the visual identity system, transportation, language assistance, and the unique and vibrant culture of China. In particular, the mobile component offers visitors a useful electronic assistant to navigate the city and manage culture shock and language barriers.

Goals

The workshop students (RedStudio) will present prototypes and design documents suggesting and demonstrating how this product might be further defined and developed. The short timeframe (three weeks) will not allow the creation of a fully functional product, but we hope that it inspires further study and development.

Solution

Adobe Flash technology offers the best platform to rapidly design and prototype the Beijing Olympics Mobile Visitors Guide. Using Flash Player to deploy the desktop web feature prototype and Flash Lite to deploy the mobile media prototype on Nokia Series 60 handsets, the students will easily integrate design and content across delivery mechanisms for a unified and integrated user experience.

Mobile Media Enabling World Class Hospitality

Everyone knows that the Chinese are famous for great hospitality, and preparing for the 2008 Summer Olympics is no exception. However, the rapid pace of development and the intricacy of the Chinese language and culture creates
barriers for visitors to enjoy their experience in Beijing. Now that most people carry a mobile phone or laptop when they travel, the Mobile Visitors Guide solves common problems for foreigners to help them communicate across languages, discover the delights of the city, manage its challenges, and more easily find their way around.

An Essential Companion

In the New China, driven by high technology and media innovations, visitors to the Olympics Beijing 2008 should enjoy digital services to help them make the most of their time here, whether on their laptop or mobile phone. Consider these modules:

• Find your way. Visually link event schedules to venue locations and find your way by subway, taxi, bicycle or walking.
• Keep score. Follow sports events by medal counts, in real-time, as winners are announced.
• Communicate. Blaze through the language barrier with common translations for useful phrases and situations on your mobile. Get pinyin transliterations, hanzi characters and audio pronunciations. Point your way out of a tricky situation using Chinese characters on your mobile’s screen. Hand over your mobile and let the Beijinger listen to your needs.
• Participate in China’s Cultural Olympics at the National Art Museum of China and use a mobile phone in an experimental large-scale group interaction within the gallery spaces.
• Explore and discover the Old Beijing and take a guided tour through the famous hutongs, an intricate maze of traditional culture and cuisine.
• Meet the Five Friendlies. The Fuwa have a great life story and getting acquainted with them will help you better enjoy their intricate symbolism which elegantly evokes the many layers of Chinese language and culture, not to mention the universal spirit of the Olympics.

Conclusion

This project prototype was presented to the Academy of Arts and Design, Tsinghua University on June 20, 2007. Relevant industry critics from Nokia and Microsoft R&D laboratories in Beijing attended the presentation.

Presentation Photos

See also

Red Studio: Teaching Design at Tsinghua↑