Today’s big announcements at Adobe MAX signal bold moves to strengthen the reach of the Flash Platform while simultaneously heading off Apple’s reluctance to offer the plugin in Mobile Safari. By allowing Flash Professional CS5 to export to native iPhone app code, Adobe wins big by giving its massive author base the means to get in on the App Store action. As a Flash developer myself who has made repeated but ultimately failed attempts to tackle Cocoa Objective-C this is very welcome news. When you’ve spent years in one language, the brain gets hardwired and is resistant to learning something so radically different. Plus, Flash developers invested so heavily in the platform, it’s a bitter pill to swallow and drop it all in favor of being locked into Apple’s ecosystem.

However, allowing Flash designers to bulld iPhone Apps means that we will see abuses of human interface guidelines and a deluge of submissions into the approval process. We can expect Apple to push back on this in whatever way they could to preserve the brand integrity of the iPhone experience.

In terms of expanding the Flash Platform into greater ubiquity, the announcements of RIM, Google and Palm joining the Open Screen Fund heralds an imminent reality where Flash content will run just about everywhere except in the iPhone Mobile Safari. By 2011, those blue question mark icons that appear where Flash browser plugin content is supposed to run will likely become a marketplace liability for Apple. Indeed, when customers can enjoy Flash enabled websites that work on every device except Apple’s, we finally imagine a scenario where not supporting Flash in Safari could actually affect sales. The demonstration of Flash Player 10.1 working very nicely on a Palm Pre underscored this notion.

Before today, I was thinking that the centrality of Flash as a de facto multimedia “standard” was in question. These thoughts were inspired by the advent of the App Store and improved web standards browser implementations such as CSS, HTML, JavaScript, and the CANVAS and VIDEO elements. However, after today, my confidence in Adobe’s ability to maintain strong support for the Flash platform is renewed by their clever responses to the shifting marketplace by resisting the force of fragmentation or at least channeling it into their advantage.

Flash haters like Gruber will remain skeptical and vaunt the merits of standards and Cocoa over Adobe’s assets. But the truth remains that there is a massive author base that knows Flash and an even larger consumer base that expects Flash content to run on their smartphones.

Video spoof of Mythbusters that Flash isn’t on the iPhone
FAQ for developers

Steve Jobs “just says no” to Flash on iPhone. Well, on first glance, that’s just what he says now, and we all know, like a good episode of Lost, there’s always more to unpack and nothing is what it seems. Considering Adobe’s product line, the so-called “missing middle Flash product” suitable for the iPhone doesn’t yet exist. The middle product refers to something between Flash Player for the desktop and Flash Lite for mobile devices. But, considering the pipeline, it’s only a matter of time before Adobe AIR Mobile hits iPhone and just about every other mobile device, smack dab in the middle of the entire mainstream interactive media ecosystem.¹

Despite all the nay-saying, Adobe seems determined to get Flash on OS X Touch.

The brilliance of both Apple and Adobe waiting for Adobe AIR Mobile to launch is that it addresses all issues and pleases both parties politically:

• AIR Mobile is likely to be built upon AJAX, WebKit and Flash Player with the ActionScript 3.0 VM, as it is on the desktop, so it will be robust, efficient, modern and support both Apple’s and Adobe’s standards. It will please developers, designers, open standards proponents, lovers of proprietary goodness, and every other regular user who just wants everything to just work.
• AIR Mobile will solve the installation and distribution problem inherent in Flash Lite. Flash Player for the desktop has never been about standalone application installation. AIR on the desktop bridges this gap. It’s only a matter of time before Adobe sends AIR into the mobile device space and allows creators to put mobile app icons on standby screens with a few clicks. Just think Apple AppStore, or jailbreak AppTapp for Adobe Mobile AIR.
• AIR Mobile will bridge the fading distinction between web – desktop – mobile by allowing creators to write software in one environment (eg. Flex Builder) and distribute codebase to all three of these crucial platforms in a truly hybrid sense.
• It’s really no coincidence that Apple and Adobe are both investing heavily in products named “Air” — the notion of the data-cloud, cloud-sourcing, everyware, and webware is nascent. Narrow-minded jargon-lovers will call it Web 3.0, but intelligent folks will hopefully leave this lame version number moniker behind and use the appropriately visionary language espoused here.

Multi-touch API is the key

The central unresolved issue that remains is a multi-touch API. Any version of Flash for iPhone will need to have its intrinsic APIs updated for multi-touch and that will need to translate to a higher-level ActionScript object so that designers and developers can trap events related to multi-touch gestures. Without gestures like pinch, flick, zoom and others, it’s really pointless to put Flash on iPhone.

At the end of the day, talk about politics, performance and battery life are probably just the red-herrings that both Apple and Adobe need to work out the vexing issue of multi-touch APIs. In fact, it will probably take a few years before all platforms (Windows Mobile, Symbian, Android and iPhone) all reckon with multi-touch on all levels of hardware and software.

The alleged YouTube Mobile Safari plugin in OS X Touch 2.0 beta is probably all the beehive needs to chill out and give Apple and Adobe the breathing room they need to get multi-touch worked out, and deploy Mobile AIR on a dizzying and divergent array of devices, platforms and crotchety carriers.

Previously:

• YouTube plugin for Mobile Safari Suggests No Flash in iPhone 2.0←
• Will iPhone Ever Run Flash?←

Footnotes

1. This is based only pure postulation and not informed by any confidential Adobe insight.

Specula-palooza Rocks On

The latest in Flash and iPhone rumor-mongering suggests a YouTube plug-in for Mobile Safari will accompany this summer’s Touch OS 2.0 update. An uncorroborated claim indicates that this plug-in is contained within the recently seeded developer iPhone SDK 2.0 beta firmware.

This would theoretically enable embedded YouTube movies to work on the billions of pages that currently flaunt the dreaded blue question mark icon. Presumably, a page with an embedded YouTube SWF player might show a thumbnail with a play icon that when tapped would load the clip in the native YouTube player or some embedded player within Mobile Safari. This would be similar to what happens when you load an embedded MP4 video file via the native QuickTime player within the iPod function.

In technical terms, Mobile Safari may parse the OBJECT and EMBED tags that point to the YouTube SWF player and redirect the path to the video into its own native player.

It’s important to note that not all YouTube videos would work. In fact, numerous clips not encoded to MP4 H.264 remain only available in the Flash codecs Sorenson Spark and On2 and therefore would not play in the Mobile Safari YouTube plugin. Although, Apple and Google seem to be colluding to convert the vast user-generated video library over into the H.263 format. It’s truly doubtful that OS X Touch 2.0 contains additional video codecs beyond H.263.

However, rather than subscribing to comment stream appearing below rumor sites, this YouTube Mobile Safari plugin is very likely not a clue towards a Flash player for iPhone. In fact, this could be the kiss of death for the chances of seeing it soon.

This kind of Mobile Safari plug-in access is precisely what Apple is shielding from Adobe (and other third-party developers, thanks to the sandbox intrinsic to the SDK). In many ways, a YouTube plugin acts as a trojan horse to usurp dominance from Flash in favor of WebKit and open standards. For most people who crave a Flash Player for iPhone, it’s the frequent dead-end to embedded YouTube clips that has them most irked. Once that kink is worked out, will users really miss Flash?

The other crucial importance of a Mobile Safari YouTube plugin is that it bypasses the main sticking point: how to implement multi-touch via an ActionScript API in the Flash Player!

We’re not suggesting Flash Player won’t ever appear on OS X Touch, but if a native YouTube plugin appears for Mobile Safari this summer, it has big implications for Adobe’s mobile strategy, and concerns the long-term viability of Flash as a de-facto standard if the mobile medium cannot be captured.

We’d love to see Flash on OS X Touch. There is a vast designer and developer community out there fluent in ActionScript that would thrive in a Touch world. It’s all about a multi-touch API from here on out. We’ve tinkered with both Mobile Safari web application design with WebKit and AJAX and also attempted to pick apart Cocoa Touch. There is no middle ground yet, and the Touch application market is nascent and fractured as a result. You’ve either got really crappy web applications or sketchy jailbreak apps. Yes, this summer’s launch of the AppStore will change the game forever. But, until the vast Adobe-enabled developer community is employed to create, the market will be constricted by limitations and learning curves.

Previously: Will iPhone Ever Run Flash? ←

On Teaching Mobile Application Design at Parsons

Dr. Dobb’s Journal, legendary software design publication, published our phone interview on its web portal yesterday on the subject of teaching mobile media design at Parsons Communication Design & Technology. John Dorsey, the editor of Dr. Dobb’s, offered probing questions that got at the nature of the program, the challenges of teaching Flash for application design, and how various tools and platforms can fit together in a technology curriculum at the service of the arts. This is all very interesting when you consider Dr. Dobb’s target audience, clearly hard-core coders and application developers. It’s a good sign when engineer-types are starting to take what’s going on in art and design schools more seriously. These distinctions are very much starting to blur.

» Interview

Web Ink for Mobile Media Curriculum

Adobe published a case study on my work with students at Parsons Communication Design & Technology in the area of mobile media design.

Specula-palooza

Risking iPhone coverage overexposure, today we ponder one of the most interesting questions about the future of Flash, iPhone and web standards. Despite assurances by Uncle Walt that Apple and Adobe are hard at work on a Flash Player for iPhone, plenty of naysayers, skeptics, and player-haters have voiced strong speculations that Flash will never appear on the iPhone for strategic, practical and technical reasons. A quick scan of comments on various iPhone related entries across the web reveals an almost universal plea amongst everyday users indicating that a dearly missed feature from Mobile Safari is the presence of a mainstream multimedia plugin. In fact, the world’s most popular piece of software in history, is well known to be absent from iPhone.¹

Mobilizing the Means of Production

Those who have written about why iPhone should not have a Flash Player don’t mask their agendas. These voices are usually programmers and developers who have always been hostile to Flash, mostly because it threatens their grip on the means of production, by bringing software and interface design to the masses. Indeed, we’ve seen the worst and the best of the web, as a result. Furthermore, because Flash has always been a mainstream, populist, albeit proprietary media format, it has been deployed for reach and ease of creation, rather than robust performance. When imagining a Flash Player for iPhone, its high-octane thirst for processor cycles does not bode well for battery life.

Monopolizing Web Standards

The second significant argument against iPhone Flash is Apple’s strategy to deploy its WebKit “web standards” platform for all third-party application development. Indeed, the recently redesigned Apple.com site has reduced its use of Flash significantly, in favor of JavaScript and browser based features, so-called AJAX. The argument goes that Apple doesn’t want to forsake its influence on the consumer-level interface design market by inserting Adobe’s trojan horse into the battlefield.² This perspective does make a lot of sense, but leaves out a tremendous amount of nuance, that we’ll investigate here.

A Muted Voice

The assumption that web standards based technology, such as JavaScript, can wholly replace Flash functionality is only somewhat true, especially on iPhone.³ Certainly, the support of JavaScript and its embrace by the web developer community over the past three to five years has changed the face of the web, earning the popular title of version 2.0. However, Flash offers some essential multimedia capabilities that JavaScript alone cannot yet offer. This includes audio support. It’s no coincidence that Bejeweled for iPhone is mute. Especially considering that many iPhone users may have stereo headsets plugged in during use, there are unimaginable uses for audio based applications, especially when combined with locative media technologies. Imagine a sort of audiopedia. The differences don’t start and end with audio support, however. Even Apple’s newly touted Web Gallery feature, part of a .Mac subscription with the iLife personal media suite needs to use Flash for its carousel photo browsing interface.⁴ Indeed, only the Flash Player offers the multimedia engine to manipulate the images as a responsive interface with the reach required on this consumer grade product.

I Want My MTV

Certainly, the most common deployment of Flash Player on the web recently is for web video. It was almost shocking to watch the FLV video format surpass RealPlayer, Windows Media and QuickTime as the most important, influential web video format, in what seemed like a matter of months, with much thanks to YouTube and others. A recent Adobe Flash Player 9 public beta featured H.264 video support, which seems part of its strategy to preserve the dominance of Flash video, especially as Apple and Google migrate towards this non-Flash based video standard. However, until the myriad of embedded SWF FLV players on perhaps billions of web pages get updated to auto-detect the client and deliver the appropriate video by codec, the web will still appear to be littered with missing plugins on Mobile Safari.

Assumptions and Speculations

Let’s proceed with the assumption that Walt Mossberg was correct, and indeed, Apple and Adobe have reached an agreement to release Flash Player iPhone in some manifestation, at some time. Of course, he could be blindly speculating like the rest of us, just running on the fact that it feels crazy for Flash not to be there. But, let’s hope he’s as well-connected and respected as they say he is.⁵ There are several scenarios for the future of Flash iPhone, which should only contribute to the over-saturated discourse by further complicating the biased opinions with an understanding of Adobe’s perspective, previously, and conveniently left out of the discussion.

A Straight Port of Flash Player 9

In this scenario, Adobe compiles the current version of the desktop Flash Player 9 for the ARM processor of iPhone. This scenario would allow all of the existing web-based Flash content to function within Mobile Safari. Authors who create content in ActionScript 3 would enjoy a noticeable improvement of performance and energy efficiency on the iPhone, since this type of content would play in the more recent Flash Virtual Machine, a marked improvement over previous runtime environments, across Windows, Mac OS X and Linux. However, the vast majority of Flash content on the web right now was created in ActionScript 1.0 and 2.0, and so does not take advantage of the improvements to the runtime. Indeed, the skeptics are correct when they assume that running web pages with Flash content, even animated banners, would put an unfortunate drain on the battery.However, what’s crucially missing from a simple straight-on port of Flash Player 9 for iPhone is a substantial confrontation with the multi-touch interface. This is likely the deal-breaker for this scenario. Although, ActionScript and Flash button symbols might offer some means of developing and designing for the iPhone’s multi-touch interface, it’s more than likely that despite the best efforts of Flash coders, the existing means to respond to mouse-based interactions will fall short of the requirements needed to respond to multi-touch. Gestures like pinch, tap-zoom, and flick are difficult to imagine as ActionScript events. Indeed, Adobe probably needs to reckon with the reality that ActionScript needs a true multi-touch API. This could be one of the many reasons why we can only assume that iPhone Flash is in development now, and may be for quite some time, especially based on Apple’s delay in releasing an official iPhone SDK.

Probability: Very Low

Flash Player 10

Looking into 2008, perhaps third or fourth quarter, we have to assume that Adobe will continue to release improved versions of the Flash Player. Version 10 is likely to provide Adobe with the required release cycle needed to fully contend with Apple’s native iPhone application API and SDK release schedule. It’s much more likely that ActionScript libraries will be written that allow true response to the multi-touch gestures, such as pinching, flicking and zoom-tapping. It’s really difficult to imagine a Flash Player on iPhone without this crucial ActionScript API.Furthermore, this allows Adobe to potentially release new versions of the Flash and Flex authoring tools that will compile in Flash Player 10, and subsequently, support a runtime environment that is tuned to the needs of the ARM architecture and precious battery life. It’s been frustrating to read the Flash iPhone haters and their blatant neglect for Adobe’s expertise in the area of building a Flash Player for mobile devices. There has simply been no mention of the possibility of Flash Lite for iPhone.

Probability: Medium

Flash Lite 2.1

Flash Lite is a very special version of the Flash Player for mobile devices, such as Symbian Series 60, Windows Mobile and others. It is not the Flash Player that typically sits in an embedded browser, like the vast majority of Flash content out there. Instead, Flash Lite content exists as standalone, full-screen, mobile applications, or more appropriate mobile media implementations such as a standby screens, wallpapers, screen-savers, and even the device’s native UI.Not only is Flash Lite compiled to the particular device, and so is limited by its processing and memory capabilities, but it also has proven to be very energy efficient, accordingly. Indeed, authors must specifically design and code for Flash Lite. It is in no way, a conversion from desktop Flash content. In this manner, designers and developers alike, are rightly forced to contend with the requirements of mobile media, in terms of interface, content and use-case considerations.It’s possible that Adobe could port Flash Lite to iPhone instead of the expected desktop Flash Player. In this regard, Flash Player would exists as a widget on the SpringBoard home-screen of iPhone, and not as a Mobile Safari plugin. Although it would not fix countless broken plugins in pages that use Flash Player, it would offer mobile media design opportunities for Flash on iPhone. Specifically, Flash Lite already offers APIs to interact with mobile phone specific features, such as triggering the vibrate, detecting battery life, and cell network signal strength. These capabilities are not offered on the desktop version of Flash Player, and may or may not be available in Apple’s official native iPhone application API or SDK. Only time will tell.In this scenario, Flash Lite becomes an avenue for designers and developers familiar with Adobe’s toolsets to create applications that exist as standalone native experiences, rather than embedded modules of Mobile Safari. In many ways, this supports the special needs of mobile media more appropriately than simply making the familiar Flash just work. Imagining a port of Flash Lite for iPhone could mean the opening of a vast market for native iPhone widget applications, designed by designers, and not restricted to hard-core Objective C programmers. Indeed, this is a threatening prospect for those that are eager to carve out a niche in iPhone application development. However, Flash Lite 2.1 probably does not offer the means to react to multi-touch gestures like pinch, flick and tap-zoom, and so it’s probably not the version we will eventually see.

Probability: Low

Flash Lite 3

Just as the desktop Flash Player is constantly updated, the mobile Flash Lite player is also expected to be evolving. It is said, that a forthcoming version of the Flash Lite player, perhaps 3, will bridge the gap between the embedded browser Flash content and the standalone mobile specific Flash Lite content.⁶ In other words, Flash Lite 3 could play not just in the mobile device’s web browser, but could also run within its native operating system environment. This jives with Adobe’s efforts to seed the use of Flash outside of the browser and distribute a desktop based native runtime, called AIR, the Adobe Integrated Runtime. In addition, a Flash Lite 3 would address the concerns of a multi-touch API, and the required energy efficiencies for battery life, memory usage and processor cycling, as well as provide ActionScript to trigger mobile device specific features intrinsic to Flash Lite.In this case, Flash would exist in two manifestations on iPhone: as standalone native applications on the SpringBoard home screen, and as a Mobile Safari plugin, playing the usual desktop based content. This gives developers, designers and end-users the best of all worlds. Flash content can be created with mobile use in mind, considering the unique user interface and energy efficiency required. Everyday users will enjoy not only the full multimedia web they’ve grown accustomed to on the desktop, but also will enjoy a market of native mobile applications that arguably, the Objective C programmers of the world, simply cannot singlehandedly service.

Probability: Medium-High

Conclusions

Flash will appear on iPhone eventually. There is no doubt that Adobe will roadmap this device into its strategy for Flash Player, Flash Lite, or both.Yes, there are significant performance, interface and user considerations that must be addressed as Flash appears on iPhone. Adobe has already demonstrated an accomplished ability to service the mobile media market. It’s only a matter of when, and what form iPhone Flash takes as it appears beneath our beloved glass multi-touch screens.One can only imagine the pressure Adobe’s product planners are putting on its Flash Player team to fit iPhone into the needs of the short-term and long-term strategy. In the short term, Adobe needs to get Flash on iPhone within the next three to six months to capture the required developer and designer audience, and compete with the ascendancy of Mobile Safari and native iPhone applications. In the long term, Adobe needs to get it right, and release a iPhone Flash Player that addresses the specific needs of both mobile media and the vast legacy of desktop Flash content out there. A premature release could spell long term disaster for Flash, as it needs to compete with the rapidly expanding Open Source and Web Standards movement. We haven’t mentioned Microsoft’s entry into the web media space with its recently launched Silverlight platform, but until we stumble upon a site that actually uses it, it’s irrelevant.Although there are many who would like Flash to just go away, because it’s not open source, not free, and tends to be used to bombard us with annoying banner ads and horrible interface design models, Flash is not going away anytime soon. However, how Apple and Adobe navigate the uncharted territory of merging mobile and desktop user experiences along with multi-touch interfaces, will certainly determine the relevance of Flash in the years to come.

Footnotes

1. See Adobe statistics on Flash Player downloads.
2. See Roughly Drafted’s analysis.
3. See A List Apart’s analysis.
4. See Cabel Sasser’s Twitter which claims dibs on this observation.
5. See the profile of Walt Mossberg in the New Yorker.
6. Speculative, but based on reliable, but undisclosed interactions with Adobe. Also, see Bill Perry’s entry on the subject.

Read Packing Lite: A mobile media interface design primer on Adobe’s Developer Center to get my perspective how to adapt to directional-pad based devices in Flash Lite. I detail the process of transferring SWF files to Nokia Series 60 devices and analyze a 4-way menu example from Adobe’s Content Developer Kit (CDK).

» Packing Lite on Adobe.com

Article to be published on Adobe Developer Center

Packing Lite: Getting Started Designing Interfaces for Mobile Media

I wrote an article/tutorial for Adobe’s Developer Center to be published at the end of August covering how to get started designing interfaces in Flash Lite on Nokia Series 60 devices. It details how to get equipped for mobile media design using Flash and the Flash Lite platform, install Flash Lite content on Nokia S60 phones with Bluetooth or Nokia’s PC Suite, discusses the unique interface design challenges, and looks at an example, highlighting the issues.

Stay tuned, as I’ll be sure to post the link to the article when it goes live. Article pushed live at 5 PM PST, August 17, 2007←

Adobe Integrated Runtime demonstrated by Big Spaceship.

This afternoon, Joshua Hirsch and Jamie Kosoy from Big Spaceship, here in Brooklyn, presented Building Adobe AIR Applications with Flash CS3 on the Adobe Connect presentation service. They did an excellent job of introducing the new capabilities of Adobe’s new desktop application delivery platform. In particular, they focused on how Flash CS3 can be used to compile AIR programs. Although still in Beta, AIR remains un-implemented in Flash CS3. The online seminar walked us through the tricky process of configuring for AIR development and introduce us to the command-line tools required to build and test AIR projects. However, Grant Skinner apparently has an extension set for Flash CS3 that assists with this process as we await Adobe’s full integration into the Publish Settings of Flash CS3. The links and notes of the presentation are posted here for reference. This post is likely to be updated and fleshed out as I process and try things out myself.

What is AIR?

AIR brings Flash and other web formats to desktop standalone applications. We should start to see more ‘downloadables’ once this catches on. Probably the best way to figure out AIR is to look at some examples on Adobe Labs→I’ve been a fan of kuler→ ever since I started following the Adobe Labs, and discovered Apollo, the codename for AIR. This is a great example of how being able to break out of the browser will encourage creative technology to engage with us more deeply.

Presentation Links

How to get started with AIR and Flash CS3

• Presentation Event Details
• About Adobe AIR
• AIR Developer FAQ
• Download AIR SDK
• Download FLEX Builder
• ActionScript 3.0 Language Reference
• Get the Flash Debug Player
• Get FlashTracer extension for Firefox
• Get Grant Skinner’s AIR Panel for CS3
• Big Spaceship’s Lab

Raw Notes from the Presentation

Here are the raw notes from the presentation. I’ll try to decipher them into more sense once I get my head around it all myself.

 airglobal.swc locations: MAC:    /Applications/Adobe Flex Builder 3/sdks/moxie/frameworks/libs/air/airglobal.swc PC:      Program Files/Adobe Flex Builder 3/sdks/moxie/frameworks/libs/air/airglobal.swc move and rename to MAC:    /Applications/Adobe Flash CS3/Configuration/ActionScript 3.0/Classes/playerglobal.swc PC:      Program Files/Adobe Flash CS3/Configuration/ActionScript 3.0/Classes/playerglobal.swc (you should make a backup of your old playerglobal.swc) AIR SDK: MAC:    Place all files in /Users/[you]/ PC:      Place files whereever you want. (I use c:AIR_SDK) Set environment variables: Right click "My Computer" Hit Properties. Go to the "Advanced" tab Click "Environment Variables" In "System Variables", under Path add path you put the sdk. If there is no Path var, you can add it. ADL syntax: adl myApp.xml Flash Tracer: After installing extension, Tools --> FlashTracer Click "Options" in the bottom right of the AddOn. PC Path MUST be: C:Documents and Settings[YOU]Application DataMacromediaFlash PlayerLogsflashlog.txt Mac Path MUST be: Hard Drive:Users:[YOU]:Library:Preferences:Macromedia:Flash Player:Logs:flashlog.txt If no Logs directory or flashlog.txt file is missing, you need to add them. ADT syntax: adt -package myAirFile.air myApp.xml foo.swf bar.swf icons videos etc