stressfree - pdf

The demise of Flashpaper sends a warning to developers

David — Mon, 08 Sep 2008 03:46:08 +0000

Do not base key functionality of your software on closed, third-party libraries - you never know what the future holds.

When Adobe purchased Marcomedia the writing was on the wall for Flashpaper from the outset. Flashpaper was Macromedia's alternative to Adobe PDF for paper-based documents on the Internet. Unlike PDF which requires a dedicated reader application (e.g. Adobe Reader), Flashpaper turns print documents into easily consumable Flash animations. Not only is Flash just as (or even more) ubiquitous as PDF, it integrates better with a user's web-experience. Consequently from Adobe's perspective letting Flashpaper live on as a potentially superior competitor to PDF on the web just did not make sense. Yet whilst this axe has been dangling above Flashpaper's head for quite some time, Adobe has only recently made it official; Flashpaper is dead, long live PDF.

This long coming announcement was a kick in the guts for businesses that have built their products on top of Flashpaper or use it for internal purposes. One bright point was that Scribd took the demise of Flashpaper as an opportunity to establish a competing product called iPaper. Whilst iPaper has some very interesting features (like integrated Google Adwords), it cannot operate 'within the firewall' on documents that are too sensitive for public release. Also iPaper's hosted architecture precludes it from being embedded into third party, redistributable applications.

All of this is a shame from a web developer's standpoint because the ability to render a document (be it PDF, Word or Powerpoint) to a slick looking Flash applet is an excellent piece of functionality. Not only does it reduce system requirements down to a Flash-enabled web browser, the end result integrates better with the browser experience. This integration leads to more possibilities when it comes to viewing and previewing the document beyond PDF's very lame ability consume an entire browser window or frame (not to mention Adobe Reader's poor load times).

The really sad part about this is that it seems Adobe is going to put Flashpaper out to pasture and simply lock-up or destroy the source code. If the product did not threaten PDF so directly it would have been a great opportunity for the company to gain some open source kudos by releasing the underlying code to the Flashpaper community. No doubt if this were to occur within twelve months there would be Flashpaper implementations in Java, .Net, Ruby, PHP and Perl. Ultimately this would have been a great thing for the industry and resulted in more Flash-content creation. Instead by circling the PDF wagons Adobe has chosen to wipe out what could have been an industry changing piece of technology in favour of some short term profits.

Unfortunately for Adobe I am sure Microsoft will not hesitate to add similar functionality to a rapidly maturing Silverlight. As a result in a few years do not be too surprised to see many sites offering Silverlight-based document viewers made possible by Microsoft-blessed (and powered) back-end tools. It will be at the point when this Silverlight-centric document future starts taking hold that Adobe execs will rue the day they terminated Flashpaper, effectively handing the 'paper web' to Microsoft.

Why Autodesk should 'Open' DWF

David — Wed, 07 Mar 2007 22:24:39 +0000

Beyond the Paper's Scott Sheppard recently pointed to McDwiff as the first partial example of a Mac-based DWF viewer. Unfortunately for the DWF starved Mac community McDwiff is simply a wrapper around a WebKit browser window pointed directly at Autodesk's own Project Freewheel web service. It fails to qualify as a true desktop application for a number of crucial reasons:

It does not (yet) add functionality beyond what is present in the Web-based Freewheel viewer.
DWF files must be first uploaded to the Autodesk web service.
There is no off-line mode or local caching to improve performance.
The lifespan of the software is entirely dependent on the existence of the host service.

Note: These limitations are not the developers fault as they have obviously only just initiated the project. It will be interesting to see where they go from here.

Looking at McDwiff and Project Freewheel got me thinking about Autodesk's direction when it comes to DWF. Unlike many AEC data format categories what constitutes the dominant 2D/3D design information exchange format has yet to be decided. However things are beginning to change with a battle brewing between Autodesk's DWF and Adobe's 3D enhanced PDF. Unlike pure data formats such as DXF, DWG and DGN goal of a design information exchange format is to provide AEC professionals with the ability to deliver digital building design information safe in the knowledge that any recipient will accurately experience the design in the manner intended by the author.

Why is such a format important?

With the rapid adoption of the Internet within business AEC professionals are conducting an ever growing portion of their design collaboration online. Central to this collaboration is the 2D documentation and 3D models that visually describe the outcomes of this design process. The unfortunate dilemma faced by AEC professionals is exactly what format to use when exchanging such information? There are a broad range of potential formats currently available but they all have their limitations as illustrated in the table below.

Format	Benefits	Limitations
Bitmap Image	Broad software support Consistent display across platforms	Limited accuracy No intellectual property protection Only supports 2D imagery
Vector-based Image	Accurate display of data Relatively broad software support Consistent display across platforms	No intellectual property protection Only supports 2D vector-based imagery
Office Productivity Document	Broad software support Relatively consistent display across platforms	Limited/no intellectual property protection Only supports 2D imagery and data
Traditional 2D PDF	Broad software support Consistent display across platforms Intellectual property protection	Only supports 2D imagery and data
Flash Animation	Supports 2D and 3D data Intellectual property protection Broad reader support Consistent display of design elements across platforms	Limited printing functionality No AEC specific creation tools
Proprietary 2D/3D CAD format	Supports 2D and 3D data	Limited/no compatibility across different software platforms Potential for inconsistent display of design data Limited/no intellectual property protection Conveys the entire digital model rather than a subset identified by the author
Standard 2D/3D CAD format (DXF, OpenDWG, OpenDGN)	Broad software support Supports 2D and 3D data	No intellectual property protection Conveys the entire digital model rather than a subset identified by the author
Propriety Building Information (BIM) Model	Supports rich AEC semantics Manages 2D, 3D and text-based data	No compatibility across different software platforms Limited/no intellectual property protection Conveys the entire digital model rather than a subset identified by the author
Standard Industry Foundation Classes (IFC) Model	Supports rich AEC semantics Manages 2D, 3D and text-based data Consistent display of design elements across platforms Industry standard	Limited software support No intellectual property protection Conveys the entire digital model rather than a subset identified by the author

From the perspective of an AEC professional the ideal format is one that can communicate 2D and 3D information that any recipient can consistently view without misappropriating the author's intellectual property.

The limitations of traditional 3D formats

Conventional 2D or 3D models cannot fill this collaboration need because they are focused on recording design data rather than exchanging design information. Information exchange is the communication of context specific data for dissemination in a certain manner, for example an A3 sheet of elevation drawings prepared to a scale of 1:100. In contrast data exchange is the communication of an entire data-set for interpretation in the manner of the recipient's choosing. There already exist a range of common AEC data formats such as DXF and the semantically rich Industry Foundation Classes. Whilst these will always be of tremendous value within the industry, when it comes to exchanging design information the malleable properties of these pure data formats limit guarantees which can be placed around communication consistency and protection of intellectual property.

The importance of consistency & intellectual property protection

Consistency is crucial in AEC information exchange because the intention of most design documentation is to act as blueprints for a physical creation. If the exchanged digital documentation is not consistently presented to all participants the design team will loose confidence in the medium and be forced to use conventional (i.e. paper-based) information exchange methods. Traditional data orientated 2D/3D formats provide no guarantee to participants that the various CAD applications available will display the parsed model consistently, if at all. To resolve this issue design information formats and their associated reader applications employ measures to ensure that information exchanged with team members is displayed in a consistent manner across all platforms.

Increasingly for many AEC professionals the digital model is the lifeblood of the design process and the source of most chargeable activities. Literally giving this model to untrusted members of the design team could compromise future income streams, devalue their role in the design process, or raise legal liability issues if the model was used inappropriately in such activities as simulation and engineering studies. Formats for design information exchange protect intellectual property by enabling the author to communicate a limited subset of the entire digital model in a manner that cannot be reutilised for anything other than its intended purpose.

The combination of these unique requirements has resulted in the evolution of two similar but competing formats from Autodesk and Adobe: DWF and PDF with 3D extensions. Whilst each have their own strengths and weaknesses they ultimately have their eyes set on the same goal: to become the dominant design information exchange format within the AEC industry. Central to success is the ability for both formats to become ubiquitous and operate above traditional concerns such as operating system, hardware device and CAD vendor.

Format ubiquity and the success of PDF in the world of 2D

Adobe's PDF format has achieved a level of platform ubiquity in the 2D documentation space that DWF and 3D PDF are striving for in the 2D/3D design information exchange arena. This ubiquity has been achieved through a couple of related factors. Firstly PDF's creator Adobe supports native PDF readers on a range of hardware and software platforms. Whilst it is unsurprising to find the Windows version receives the most attention, users of other platforms are not left feeling isolated when it comes to inclusion of new features and ongoing support. Secondly a rich developer ecosystem has increased the acceptance of PDF from merely a proprietary (yet open) format, to a vendor neutral and soon to be internationally standardised platform for business information. There are literally dozens of non-Adobe sponsored, open and closed source implementations of PDF libraries, readers and writers written for a variety of different computer languages and applications. This diverse ecology ensures that even if Adobe were to pull support for some, or all aspects of PDF the format itself would remain viable for years to come.

Time will tell whether Adobe's 3D extensions to PDF will receive the same platform and developer support as its older 2D equivalent. Considering the 3D extensions are being standardised during the forthcoming ISO certification process there stands a very good chance of the extended format gaining broader platform and developer support in the near future. Considering PDF's current momentum DWF faces a difficult challenge if it wishes to become the dominant design information exchange format. Fortunately for DWF's chances its creator Autodesk is not idle when it comes to this challenge.

Autodesk's DWF initiatives

Unfortunately for Autodesk DWF has yet to reach the same level of platform nirvana held by 2D PDF. However they do have five initiatives underway to encourage adoption of the format within the AEC industry:

Extensive DWF support is built into many existing and new Autodesk products.
A revised DWFx format has been released which is built on top of Microsoft XPS technologies. This benefit of this format is that is can be read by Windows Vista without the need to install a dedicated viewing tool.
Recently Autodesk Design Review was released for free alongside existing DWF Viewer and DWF Writer applications for Windows.
A DWF toolkit has been made available for software developers on Windows, Mac and Linux to write for and integrate with DWF applications.
Within Autodesk's labs they are working on Project Freewheel, a Web-based, hosted DWF viewer.

Gauging success through market significance

Unfortunately for Autodesk the ultimate success or failure of DWF will not hinge on how well it is implemented within their own product line. Instead the deciding factor of its success will ride on its level of uptake within the AEC industry when compared to the use of PDF and the myriad of other traditional format options. Market significance will depend on three equally important factors:

The ability of Autodesk to deliver compelling reasons why AEC professionals should use DWF instead of PDF or traditional design data files.
The capability to read DWF in all design environments on any potential platform.
The proliferation of DWF beyond an Autodesk-only software mindset. In essence the concept of DWF must become 'bigger than Autodesk'.

If Adobe can leverage PDF's existing user and developer base to become the mainstream 2D/3D design documentation format Autodesk risks losing one advantage it has held within the AEC industry; dominance in the file format arena. In the past Autodesk has used its control of the legacy DWG format to assist software sales and deter competition in the 2D/3D CAD space. Two such examples of this have been the encryption applied to DWG by AutoCAD 2004 and more recently the copyright lawsuit filed against the Open Design Alliance for their support of TrustDWG. Undoubtedly whichever vendor can achieve a similar level of format dominance in the design information exchange field will hold greater leverage when it comes to future sales, marketing and development activities.

Initiatives that appear to be working

Two of the five Autodesk DWF initiatives appear from the outside to be working very well. Integration of extended DWF capabilities within a range of Autodesk applications is signaling to the industry that not only are Autodesk serious about DWF but it is also a very capable format in its own right. The availability of free reader and writer applications ranging from the complex but very capable Autodesk Design Review down to simple DWF reader/writer tools are also encouraging users who do not use or have access to the latest Autodesk applications to experiment with DWF on a limited yet worthwhile basis.

DWFx: clouding the water or lighting the way?

Of the five initiatives I am least certain about the establishment of DWFx, the derivative of DWF based on Microsoft's XPS (a.k.a. 'PDF killer') technology. The problem DWFx poses is that it makes the process of explaining to AEC professionals what DWF is and why it is useful more complicated. Rather than having one format to choose from there is now two that do exactly the same thing but have varied levels of support in different software versions and operating systems. Whilst there are some fairly valid technical decisions behind the establishment of DWFx its timing seems geared more toward product release cycles (i.e. Windows Vista) than any industry call for a DWF successor at this point in time. The telling question which remains to be answered is how the consumers of this technology react (if at all) to these diverging DWF formats, especially considering the imminent entrance of 3D PDF into this field.

Freewheeling Freewheel and its effect on the developer community

After looking at Project Freewheel for a while I have come to the (maybe wrong) conclusion that it is an excellent concept that in the long run is actually doing more harm than good to the DWF developer ecology. What Autodesk should be focusing on is providing application and Web developers with a tool-set to create their own Project Freewheel rather than building functionality on top of it. McDwiff is an obvious example that the current DWF developer toolkit provided by Autodesk is not what high-level application developers want. What is needed is an easily deployed DWF engine available for any platform that those interested in working with DWF can quickly utilise in a few lines of code. Ironically Project Freewheel provides this functionality but its ties to a central website limits developer potential.

How can Autodesk provide these development tools when they do not have the resources or experience to support multiple platforms? The answer is they cannot, but what they can do is encourage other developers to help them out in achieving this task. Although the DWF developer tools are available free of charge they are not easily accessible under a common license. Instead developers are forced to complete a rather large form and agree to a license which essentially states you can do anything with the code as long as you acknowledge the changes made when distributing the software. Forms and unusual licenses are barriers to developers, especially those used to working in an open source world of anonymous CVS and free (as in freedom) license models.

With the kit already available without cost under a relatively open license agreement why not take the next step and establish a SourceForge project for the software and release it under a commonly understood (LGPL or Apache) free license? After all it is not uncommon for large proprietary software companies to do such a thing. The benefit of the move would be to send a clear sign that Autodesk is serious about fostering a developer community. This extended developer community would be free to create DWF-centric applications and fork or port the existing DWF code-base in a variety of ways. The short term benefit of this action is that the DWF toolkit would be turned from an isolated download into the basis for an entire online development community similar to that of the Open Design Alliance's OpenDWG and OpenDGN initiatives. In the long term the opening of the DWF toolkit could be used to signal the tip of a very large, open source DWF iceberg.

Autodesk should not simply stop at freeing up distribution of their DWF toolkit. Instead this action should be used as the prelude for the open sourcing of the DWF Viewer and Project Freewheel code bases. With Autodesk Design Review now available free of charge Autodesk's DWF Viewer is of no real value from a commercial product perspective. If opened to third parties its code-base could be of tremendous value as a starting point for DWF readers on other platforms such as OSX and Linux. Whilst it maybe difficult to believe there are communities of developers that take pleasure in getting Windows centric applications like DWF Viewer running on multiple platforms. Likewise the goal for Project Freewheel should be to act as an open technical demonstration of what can be achieved with DWF on the Web. Whilst it is easy for Freewheel to be viewed as a potential money maker for Autodesk in reality the concept's true value will only be realised when third-party developers can embed similar functionality within their commercial or in-house developed Web applications operating behind corporate firewalls. Consequently instead of productising Freewheel, Autodesk's goal should be to inspire Web developers to write similar DWF manipulation functionality in PHP, Java, Ruby and other Web languages which may evolve. It is only through actions such as this that Autodesk will be able to proliferate DWF technologies rather than simply monopolising their use in the interests of short term profit.

Conclusion

The looming DWF vs PDF format debate will be beneficial to the AEC industry as the heated competition will no doubt enable new levels of digital collaboration. Currently it is difficult to say which format will become dominant but undoubtedly whichever vendor fosters the strongest developer ecosystem will hold a significant advantage. With this factor in mind Autodesk should be actively nurturing the growth of its external DWF developer community by removing of all barriers to entry. This could be achieved through the release under a common open source license of a select few DWF-centric technologies within a community friendly environment such as SourceForge. Through this process Autodesk would stand to grow an even stronger DWF developer community and in the process improve DWF's chances of becoming the ubiquitous design information exchange format.

Adobe releases PDF standard for ISO certification

David — Wed, 31 Jan 2007 09:46:25 +0000

On the 29th January Adobe announced it will be seeking ISO certification for the PDF 1.7 standard. Up until this point many vendors have been able to implement the PDF standard based solely on trust that Adobe will not significantly change the format and break their respective implementations. If PDF gains ISO certification then this will ensure any vendor can develop for and use the standard in the knowledge that it will not change and be 100% interoperable with other implementations of the same standard.

This is good news for governments and business as it means once certified PDF will become a permanent, unencumbered format. These characteristics will enable organisations to use PDF as an archiving medium for 2-dimensional digital documents with the confidence that no single company can dictate or control the use of the format. This is important because in the past companies like Microsoft have unduly effected the industry with their monopolistic control over formats (as evidenced by the infamous Halloween Memo).

Microsoft and Autodesk will not be too pleased at this development as both parties have formats which are competing against PDF (XPS and DWF) for user acceptance. What is not so clear from the announcement is whether Adobe's ISO ambitions cover their newly announced 3D extensions to PDF or just the well entrenched 2D format. My guess would be this week's announcement only covers 2D as 3D is no doubt still under constant development. Either way it adds pressure to Autodesk who are really pushing DWF hard as the most important medium for 2D and 3D documentation in the Architecture, Engineering and Construction space.

Deelip Menezes posted the ISO news on his blog and he got an interesting reaction from Brian Mathews of Autodesk Labs. In the comment he argues that DWF holds the advantage because it is based upon Microsoft's XML-based XPS format which is more clearly documented and more 'open' because it is in XML. This "XML good - binary bad" line of thinking ignores the fact that XML by design is significantly slower and heavier than a binary format which limits its applicability. Also it is important to remember that a format's clarity of documentation is entirely dependent on the efforts of the format creator, something Microsoft has proven notoriously bad at.

Brian also raises another point:

"PDF has always been an open specification, but has been based on a very complex binary format that takes 1000's of pages to fully document (with lots of external reference documents)."

Out of interest I downloaded the Adobe PDF specification from their website. Yes it is 1,310 pages long but this includes almost 200 pages of index and 100 pages of examples. That leaves a round about 1,000 pages devoted to the actual standard and from a quick click through I found the sections well laid out and in parts quite an interesting read (well as interesting as a standard can get). Whilst 1,000 pages is a lot there is a lot to cover and I would rather see a format span 10,000 pages and be understood in depth rather than glossed over in 500. Also another yardstick by which a good standard can be judged is how often it is independently implemented and for PDF this number is quite significant, just checkout Wikipedia's rundown.

The other issue Brian Mathews raised was Adobe's XML-based PDF format codenamed Mars:

"Adobe followed recently with an announcement that they too will be developing their own fully XML alternative to PDF. Why? Because PDF isn't open enough since it isn't XML."

Reading through the intentions behind Mars it becomes apparent that it is not so much a "PDF replacement" as Brian implies but rather a complimentary format. Why? Binary formats are difficult to index by search engines or pull data out of without pre-processing. Mars is not a rewrite of the PDF format but rather an implementation of the PDF standard using XML syntax (after all, XML is just a language syntax). This format enables quick, low overhead indexing and data manipulation at the cost of end-user speed. As a consequence Adobe are pushing this as a complimentary format and certainly not as a replacement to binary PDF. Employing complimentary file formats is not unusual, for example the Industry Foundation Classes come in both binary and XML versions to suit specific use cases.

What this all this means in the long term is still uncertain but it certainly looks like the competition is heating up.

Great little tip from MacWorld

David — Tue, 17 Jan 2006 19:40:11 +0000

Most of the tips found in MacWorld are fairly dull but this one about setting up PDF Services is pretty cool. It turns out in OSX 10.4 the Save as PDF button is customizable so you do not have to follow the instructions in this article, just click on the Save as PDF button and then select Customize from the menu. It would be nice if you could chain commands together in this menu so with one click you could compress, encypt and save receipts to a designated folder.