WWDC 2007 WebKit SVG Demos

Following an invitation from Apple, Dean Jackson and myself joined the crowds of developers who participated at WWDC 2007 to co-present two sessions about the latest advanced of the WebKit Web rendering engine, focusing first on vector graphics and then on rich media. Unfortunately, all the contents of the sessions were under the WWDC NDA and we won't be able to discuss the exact points that were delivered to the audience. We can however share the demos that we presented there as these do not require any specific software covered by the NDA, indeed they'll run just fine in a recent WebKit nightly or even the recent Safari 3 beta for Mac OS X. The demos presented were:

• A scalable search box user interface widget combining XHTML, CSS and SVG
• Animated graphs generated from SVG.org Google Analytics data feeds using AJAX and SVG
• Rich media Web interface for a Delicious Library database using XHTML, CSS and SVG
• A simple set of introductory demos to the <canvas> APIs

We will be putting up more information, such as screencasts and cleaned-up code, in the future but in the spirit of getting something out there in a timely manner, you can now get most of the demos wrapped in a single archive with all the code and various data resources. The code is covered by the MIT license, for more details see the LICENSE.txt file in the package. Also, make sure you read the README.txt as some demos require you to do a tiny bit of work to get them to work.

Feel free to use the commenting system to share any issue that you may have with this code, Dean and I will make sure anyone who wants to check it out actually gets the code running. Shake and bake!

Speaking Engagements

Following January's Mobile SVG extravaganza in New York City, I'll be speaking about SVG a few times in the coming months. First off, I'm flying out to Sydney to attend the first W3C SVG WG meeting of 2007, and incidentally speak briefly about Rich Mobile Experiences with SVG at MoMo Sydney, where some fellow Working Group members (Nandini Ramani from Sun, Andrew Emmons from BitFlash and Andrew Sledd from Ikivo) should be lending a hand presenting some very nice services that have been shipping in the last few weeks. Fast forwarding in May, I will be presenting about a session called Putting SVG and CDF to Use in an Internet Desktop Application at XTech 2007, the European XML Conference, held in Paris, my hometown. This talk will be about our experience with front-end XML languages in Joost:

The goal of this talk is to present how client-side XML technologies (SVG, (X)HTML, XUL, CSS, RDF, DOM and ECMAScript) were put to use to create a killer, multi-platform desktop application built around the Internet allowing television-watching via peer-to-peer networks: The Venice Project. The main points of this presentation will be to illustrate how the various XML grammars were put to use for different tasks, all within a unified XML presentation layer:

• SVG, DOM and ECMAScript for finely tuned, animated and highly interactive user interfaces that scale gracefully to any resolution and screen aspect ratio
• HTML, XUL and CSS for flexible control of the display of text content coming from remote data sources
• RDF, SPARQL and remote requests for data retrieval

The common thread within this talk will be to show as well that this technology mix is directly applicable within browser-based Web 2.0 applications as well.

There is another amazing event at which I'll be appearing in June, but it's too early to give out specifics, so I'll save that for a future post.

UPDATE: The MoMo Sydney website has a wrap-up of the event with pictures.

The Venice Project is Now Joost!

Today was a big day at work as the code-name we operated under for a year now, The Venice Project, was put to rest for the official brand name that we will be using from now on: Joost! This does not mean that we're quite ready to let the cat entirely out of the bag about our "TV meets the Web" venture: we are still in a closed, invite-only beta program for a while. However, to celebrate on this auspicious day, I will send an invite to the first few people who drop a comment here with their first name, last name and email address. Also, now that we have reached a more public level, I will try to post about some of the cool things that we are doing with SVG and other sweet XML technologies in the client application, which is the part of the application I work on.

Sorry, no more invites for the moment.

Getting an SVG Element's Full Bounding Box

Getting the fully-transformed bounding box of a given SVG element is something quite handy and not given for free by the SVG DOM right now. However, the SVG DOM provides all the required foundations to compute such a bounding box, by providing the SVGLocatable::getBBox(), SVGLocatable::getScreenCTM() and SVGPoint::matrixTransform() methods mostly. So, with no further ado, here's the code to a utility getScreenBBox(element) function:

function getScreenBBox (element) {

  // macro to create an SVGPoint object
  function createPoint (x, y) {
    var point = document.documentElement.createSVGPoint();
    point.x = x;
    point.y = y;
    return point;
  }

  // macro to create an SVGRect object
  function createRect (x, y, width, height) {
    var rect = document.documentElement.createSVGRect();
    rect.x = x;
    rect.y = y;
    rect.width = width;
    rect.height = height;
    return rect;
  }

  // get the complete transformation matrix
  var matrix = element.getScreenCTM();
  // get the bounding box of the target element
  var box = element.getBBox();

  // create an array of SVGPoints for each corner
  // of the bounding box and update their location
  // with the transform matrix
  var corners = [];
  var point = createPoint(box.x, box.y);
  corners.push( point.matrixTransform(matrix) );
  point.x = box.x + box.width;
  point.y = box.y;
  corners.push( point.matrixTransform(matrix) );
  point.x = box.x + box.width;
  point.y = box.y + box.height;
  corners.push( point.matrixTransform(matrix) );
  point.x = box.x;
  point.y = box.y + box.height;
  corners.push( point.matrixTransform(matrix) );
  var max = createPoint(corners[0].x, corners[0].y);
  var min = createPoint(corners[0].x, corners[0].y);

  // identify the new corner coordinates of the
  // fully transformed bounding box
  for (var i = 1; i < corners.length; i++) {
    var x = corners[i].x;
    var y = corners[i].y;
    if (x < min.x) {
      min.x = x;
    }
    else if (x > max.x) {
      max.x = x;
    }
    if (y < min.y) {
      min.y = y;
    }
    else if (y > max.y) {
      max.y = y;
    }
  }
  
  // return the bounding box as an SVGRect object
  return createRect(min.x, min.y, max.x - min.x, max.y - min.y);
}

Adobe Apollo's Technology Mix: Nothing New

Last month, the first incarnation of the MAX conference under the Adobe took place, and Adobe's new Apollo runtime was the hot ticket that was previewed to developers, in preparation for the first public alpha coming up later this year and an anticipated launch of late 2007. For those who haven't heard of Apollo yet, Apollo is the code-name for Adobe's next generation runtime that basically allows developers to seamlessly combine two worlds that were mostly living in isolation of each other: HTML and Flash.

I actually was privy to work being conducted at Adobe prior to the announcement of the Macromedia acquisition, when Flash was still a technology non grata within Adobe. Times change. I was consulting on Adobe JamJar, a cool collaborative workspace project which was developed in SVG, and has since been reworked as a Flex application and is available as an alpha on Adobe Labs. While we were developing the basis for JamJar, we were also keeping tabs on the latest developments within Adobe's Advance Technology Group about mixing HTML and interactive, rich graphics together in a next-generation runtime. You can safely bet it didn't involve Flash at all. I can't go into much detail, as this information is confidential, but the concepts of Apollo were already firmly in place, but the exact technology mix was something completely different.

The new Adobe had been building a buzz around Apollo prior to MAX already, but this conference was the first time Adobe actually gave details on how developers will be able to work with Apollo. It is of course too early to make any kind of definite statement about Apollo successes or failures, but I'd like to weigh in on the two big take-away items in Chris Berchdorf's presentation on HTML integration in Apollo.

According to Chris Berchdorf, the first major take-away is that Apollo introduces a unified rendering layer for Flash and HTML content. Indeed, this is a big deal as it has always been a pain to mix and match HTML and Flash in the same Web application where Flash content would mostly be something you'd have to reference as a black box with an <object> element. There are variations to that of course, for instance the Flash plug-in can be instantiated with a transparent background which allows to have Flash content blend in nicely on top of HTML, but this has always come as an expense in terms of rendering performance. Additionally, a big issue was never, I think, resolved: rendering HTML on top of Flash content. To be fair, all of this is a shared issue of media types outside of those natively handled by a Web browser that have to integrated via a plug-in.

The scoop here is that soon, with Apollo, a single top-level rendering and compositing model will allow for all types of subtleties in how developers wish to organize HTML and Flash contents, catering for all grievances listed above. This is great news for the Flash developers, as Adobe have come to the realization that Flash cannot do it all on its own, and that certain parts are much better done in HTML and CSS. This includes, for instance, anything involving automated layout and rich text content. But, from a technological point of view, there is really nothing new here, nothing ground-breaking.

During his presentation, Chris Berchdorf showcased the new Apollo rendering capabilities by layering a Web browsing view on top of Flash-rendered contents. During the demo, the Web page was rotated, scaled, blurred and alpha-composited. This rings a bell, something we've seen demonstrated last year in an open source and publicly available codebase: Mozilla. Indeed, at XTech 2005 event (about 18 months ago), developer Robert O'Callahan showed off the then latest graphics advances in the Mozilla platform -- on which the hugely popular Firefox browser is built -- and one of this demonstration was the rendering of (X)HTML content, or entire Web pages, inside an SVG <foreignObject> element. Using this mechanism, Robert could apply all kinds of built-in SVG transformations and visual effects to the Web page: scaling, rotation, alpha compositing, blurring -- you name it. The high-level capability shown in these Apollo pre-alpha demos and those shown in Mozilla 18 months ago are the same: mixing and matching of traditional HTML and CSS content with rich graphics technology in a single rendering layer. You only have to swap Flash SWF, a proprietary binary format, with SVG, an XML-based, royalty-free language.

While this shared rendering layer for HTML and SWF is interesting, and very much a revolution in the Flash space, there is just no major achievement in the grand scheme of things here. And I think we can safely say that integrating (X)HTML and SVG together is going to be more natural to developers than putting together HTML and SWF content, given both (X)HTML and SVG are both expressed in plain text markup, literally cut from the same cloth.

Now, onto the second big take-away items from Chris's presentation: the JavaScript and ActionScript programming models are now completely bridged. Indeed, similar to the limitations in visually mixing HTML and Flash content, programmed communication between JavaScript (HTML) and ActionScript (Flash) has always been severely limited by the plug-in architecture. For instance, for an ActionScript object to be exposed in the JavaScript context, it had to be serialized to XML and then recreated as a JavaScript object. Soon, Apollo will make all of this go away, and programming across JavaScript and ActionScript will be easier than ever.

Once again, this is a big win for the Flash developers. And once again, there is nothing new here from a technological point-of-view. The API used to program HTML and CSS is the DOM. These same APIs can be used to program against any kind of XML markup, such as, for instance, SVG. In a compound XHTML and SVG document, developers can use one single set of scripting APIs, all in one context, to program both HTML and rich graphics (SVG). This is what works today in browsers such as Firefox (since version 1.5) and Opera (since version 9), and already in development builds of WebKit, the Web page rendering engine used by Apple's Safari. Both Google Maps and Windows Live Local use this functionality.

Here again, the integration between XHTML and SVG outshines the HTML and Flash mix. Even though ActionScript is based on JavaScript, the APIs you actually use to program your Flash content have nothing in common with the DOM, they are two very different things. In a compound XHTML and SVG document, you only use one set of APIs, the DOM.

Additionally, I'm wondering how Apollo will allow the kind of JavaScript programming used today, which is loosely-typed, to be used with the far more advanced incarnation used in Flash 9 which is strongly-typed and much more advanced feature-wise. Adobe have been pushing hard for Flash developers to use the latest and greatest ActionScript language-level features, but now, they have to integrate in a world that's not quite so cutting-edge yet.

Another thing that I'm wondering about is duplication of APIs. So far, Flash has lived in its own world, where it was tough for developers to leverage the host environment (HTML/JavaScript) capabilities. But the same way that Adobe have pushed for more and more application-level APIs in Flash, the browsers have been adding new APIs, and are still in the process of doing so, as work in the W3C WebAPIs Working Group and in the WHATWG shows. Which APIs will developers use for networking, drag-and-drop, cut-and-paste, etc.

My goal is not to lessen the achievements that Adobe is the process of making with Apollo, but rather to underline that the mix of HTML with real-time, rich graphics is not easiest done with two unrelated technologies as it would be with technologies built on the same grounds: HTML and SVG. Certainly, there is nothing new offered by Adobe here, and I would bet the old Adobe would have gone for something more natural. Hopefully, the old Adobe still lives, and Adobe Mars uses SVG as a complement to XHTML and CSS for its PDF-in-XML initiative.

A Look Into SlingPlayer Mobile's Use of SVG Tiny

During the annual Symbian show in London two weeks ago, Sling Media introduced their Sling Player for the Symbian mobile platform, the foundation for the S60 and UIQ platforms. An interesting fact about the SlingPlayer for Symbian is that SVG Tiny, SVG's mobile profile, is heavily used to power the user interface. I had the chance to exchange emails with George Williams, an old acquaintance I made back when I was consulting for Adobe, who is now leading the efforts for interaction design of the SlingPlayer on Symbian and provided me with information as to how SVG was used in the creation of this application.

All the graphics in the SlingPlayer for Symbian user interface are rendered using SVGT 1.1 Plus, as supported by the Scalable UI APIs in the S60 platform. These APIs really fit the needs of this application as they allowed them to have code that would load SVG assets or bitmaps using the same APIs, as the Scalable UI APIs can equally load vector-based SVG assets or bitmaps. That came in very useful as the SlingPlayer deal with bitmaps when it comes to channel logos, and given they have to manage hundreds of them and that they all come from bitmap sources, this was a real win compared to having to convert all of those to vectors.

To create the graphical assets for their UI, George's team used a variety of tools to fine tune the icons as they wanted them. First, they mocked up their desired UI in Photoshop, then used Illustrator CS2 where they spent most of their production time designing the vector assets. Following that, they went into GoLive CS2 to validate some of the more complex graphics and finally used Ikivo Animator for some remaining graphics cleanup. Once all of this was done, the design team used the Nokia Series 60 Theme Studio to compare the SVG graphics against the original Photoshop skectches. Adobe have some information on their website about using their Creative Suite with Ikivo Animator for mobile content development.

At the moment, none of the SVG assets used are animated, but they do make use of transparency, gradients, and other more advanced graphics features of SVG Tiny. The S60 Scalable UI APIs actually don't currently allow for loading animated contents, so that fits right with the current state of the technology they chose to use. However, George tells me that his team are investigating how to leverage SVG Tiny 1.2 for some future products. Given the tight integration of graphics and multimedia in SVG Tiny 1.2, this is only natural and should allow them to evolve from the existing SVG assets they are currently using.

I asked George if they ever considered Flash Lite as an enabling technology, or even as a prototyping tool, and it turns out they never really considered using Flash Lite as it didn't have the integration capabilities that Scalable UI APIs offered, mainly mixing and matching raster and vectors as assets. In fact, this just confirms a key point I usually put forward in talks about Mobile SVG in that the key differentiator between SVG Tiny and Flash Lite is not so much to be found in what graphics, interactivity and multimedia features they offer, but rather that SVG Tiny is integrated tightly in many of the application layers of mobile devices. Indeed, you'll find SVG Tiny supported natively in browsers, UI toolkits, programming environments all the same.

This is the first of a series of usage of SVG in real-world applications that I hope to be able to highlight on this blog in the future, but this definitely is a fine example of how to put Mobile SVG to use today for your mobile application development.

Advanced SVG Animation Techniques

That's the name of the latest article that was just published on the new Dev.Opera, Opera's latest developer-oriented website providing high-quality articles on advanced web design topics, such as mobile, JavaScript, widgets, voice, and of course SVG! In fact, the SVG section is already well represented with 4 articles! Opera are really putting their money where their mouth is with SVG, by not only having the most advanced native support for SVG in any browser and offering the first widgets framework to support SVG rendering, but also promoting the technology relentlessly throughout the community. I'm really happy they called me up to contribute to this website and I'll make sure to contribute more articles. And don't forget to comment on my article on the Dev.Opera forums!

Next Generation Mobile SVG for Everyone

A little over a month ago the SVG Tiny 1.2 specification became a Candidate Recommendation, one of the final stage of development of a W3C specification where the technical work is expected to be frozen and mostly complete and implementors are requested to send their implementations in order to check the technical validity of it all. As the press release states, SVG Tiny 1.2 is well on track to be adopted across Mobile Industry bodies such as OMA, 3GPP and OMTP. Also, for the last 18 months, all interested parties in the industry, such as carriers and terminal manufacturers, have all been evaluating how best to use SVGT 1.2 in their products. However, when it comes to smaller organizations such as mid-size content providers and small mobile design studios and application development houses, SVGT 1.2 has been locked away.

Even though SVGT 1.2 is a royalty-free and open standard, most of the actors in the mobile field implementing it are private companies building closed source software, such as BitFlash, Ikivo and Opera. As a matter of fact, open source is not such a common ticket on mobile devices as it is on desktop computers today, and besides a couple of Linux-powered devices from Motorola and Nokia and the WebKit-based S60 browser, mobile phone software is virtually always closed-source. Further than that, the vast majority of mobile phones are based on completely closed platforms (RTOS) that prevent developers and third parties to developer native software, and J2ME is usually the only option, apart from S60. So, in my experience, most developers and content creators out there wanting to give new mobile technology a shot use S60 devices with beta software.

This approach is available for SVGT 1.2 today, but only to people who have the necessary structures to enter in agreements with BitFlash or Ikivo. This means that your organization has to have an interesting profile for them to justify the resources providing support to you while you evaluate the product, as well as having the right legal people in-house to enter such agreements. Basically, if you're just some guy out there with an idea and a will to experiment, you're likely to be on your own.

I fully understand why BitFlash or Ikivo choose to work that way though. The main reason I believe is that they need to control who gets to run their code in order to manage expectations and make sure no one jumps to the wrong conclusions with regards to their SVGT 1.2 implementation. Indeed, SVGT 1.2 has been so far in heavy development, and implementations of it were constantly being improved and changed to match the latest changes in the specification. Just releasing this stuff into the wild could have had very negative impact to the untrained eye where people could have just lynched precious work for some odd incompatibility with the unstable specification.

Now that SVGT 1.2 has entered Candidate Recommendation, and can thus be considered stable, it is time for all this to change. SVG Tiny 1.2 has an incredible feature set, and very big improvements over SVGT 1.1, much, much bigger than a 0.1 version number jump would suggest. The most obvious features are scripting with ECMAScript, audio and video support, fully synchronized with SVG animations, wrapped text within rectangles, gradient fills, transparency, etc. This new feature set is a big deal and opens up the door to a wide array of new usage for SVG on mobile devices and everyone out there should be able to try it out.

Flash Lite 2 and SVG Tiny 1.2 can easily be considered as tie-breakers on a feature-per-feature basis. The downside of Flash Lite 2 is a general lack of support from mobile industry bodies, no integration with J2ME and, so far, an insufficient backing from manufacturers. But if there is one place where Flash Lite 2 fares better currently is that an SDK for it is readily available to everyone who wants to experience that technology today. I really think BitFlash, Ikivo and any other fine purveyor of Mobile SVG software need to make a move now so that SVGT 1.2 fully has the shot it deserves to be the number one rich media technology on mobile phones in 2007 and beyond. Content Development Kits, or CDKs, should be made available to small-size content and application developers with a simple EULA and software for Windows (supplementary platforms would be even better) and S60. As for providing support for these CDKs, I think we can anticipate strong communities to emerge as soon as these CDKs become available and limit the efforts needed to get successful programs running.

Technology speaking, SVG Tiny 1.2 goes a long way towards what next-generation mobile services providers need, and the ball today is in the court of companies such as BitFlash, Ikivo, Opera and others to bridge the technology gap and provide evaluation players to hungry developers.

Adobe kill their SVG Viewer all too soon

After years of nothing much going on around it, the venerable Adobe SVG Viewer is now officially going to pass away as Paton Lewis, a veteran of the Adobe SVG team, just announced it on the Yahoo! Groups svg-developers list. Adobe also posted a short FAQ detailing the process of the end of life of ASV and the key points are as follows:

• Support for ASV will cease on January 1st, 2007 and no more security patches will be issued
• There are know issues with running ASV on Vista, and there will be no steps taken to ensure certification
• Distribution via the Adobe website is terminated as of January 1st, 2008
• Further distribution will be allowed on intranets, but not on public sites, as per the EULA
• Adobe does not have plans to release the source code for ASV
• Existing SVG support in authoring tools is not under reconsideration

In today's situation, the gist of it is that there will be no significant support for SVG within Internet Explorer come 2008. There have been strong rumors of Microsoft moving in the direction of supporting SVG natively in future versions of Internet Explorer beyond version 7, but there are no firm public statements as to when this support would materialize, and 16 months from now might well be too soon for Microsoft to unveil this anticipated support. The main problem is that no decision maker out there can afford to place a bet on such potentialities, and the only real thing right now is that SVG support in Internet Explorer is going away fast.

Now, the fact that Adobe has decided to publicly announce the end to their efforts supporting SVG on the client is no suprise and can in fact be welcomed as a sign for the entire community to focus on native support of SVG in the browser, which is a task advanced at varous stages for all of the major alternative browsers (Firefox, Opera and Safari). The problem is that, to be blunt, Adobe is dropping the ball on this affair by pulling support and distribution much too soon. Adobe SVG alumnus emeritus Jon Ferraiolo, often regarded, and rightly so, as the godfather of SVG, and now leading the Open AJAX initiative at IBM, has posted a very spot-on response to his former's employer's announcement. Here are the key points:

• Instead of mere months, Adobe should provide at least 2 to 4 years of continued support for the product
• Adobe should donate the high-level source code for the viewer to the open source community
• But the most irresponsible part is to prevent any further distribution in 16 months

The last point to me is the key issue. Why would Adobe not continue distributing the plug-in via their website for years to come until native support in Internet Explorer is reliable enough or a viable plug-in alternative materializes. What would be the problem in just hosting a simple 3MB downloadable with just warnings about some of the known issues with Vista support? This move shows that further than officially telling the world that ASV development is over, Adobe is really getting out of its way to actually prevent the SVG ecosystem to peacefully get on with its life. If I had to choose one part for Adobe to reconsider, it would definitely be that one. Waiting for 2009 seems an absolute minimum before halting the distribution of what, today still, remains one of the most advanced solution for rich graphics in the browser. It's no secret that, following the acquisition of Macromedia, Adobe has placed its eggs in another basket in terms of standards-based client technologies, but such an abrupt termination of service is a very inconsiderate move from a company that despite its shortcomings, and missed opportunities regarding SVG, has always been supportive of SVG's development on the internet and in more specific industries.

At this point, it is not clear to me at all whether Adobe will be open to changes and discussions about how it proceeds to lay ASV to rest. But the entire community should come together and send a clear message to Adobe that they are going too far and that these terms are unacceptable to the community at large. Meanwhile, we can all continue to applaud the great progress made across the last few months by Opera, who have tremendous SVG Basic 1.1 support across their entire offering with their Opera 9 product line, the Mozilla community gearing up for a fantastic Firefox 3.0 release in the first part of 2007 and Safari's WebKit rendering engine progressing every day. Now is also a good time for Microsoft to step forward and show their dedication to supporting and advancing the core, shared set of Web technologies that they have been advocating and show Adobe that standards-based rich graphics have a flourishing future ahead of them.

UPDATE: Given the feedback from the community asking for more flexible dates for both end of support and end of download, Adobe have pushed back the dates to, respectively, January 1st 2008 and January 1st 2009. This is a much more reasonable timeframe and reflects better on Adobe's business practices.

UPDATE: With the continuous pressure from the community, Adobe decided to extend the distribution of their SVG Viewer indefinitely:

In response to customer feedback, Adobe has extended download availability once again for Adobe SVG Viewer.

Adobe does not currently have plans to remove Adobe SVG Viewer from the Adobe.com download area. Adobe recognizes that customers have built Web applications that depend on ASV being available for download, and although Adobe does not plan to develop ASV further, we plan for the existing versions to be available for download as long as our customers rely on them.

For the official announcement, please see http://www.adobe.com/svg/ eol.html