ODF and OOXML: Something New to Ponder
In a couple of weeks is the Government Open Source Conference (GOSCON), to be held in Portland, Oregon on October 15-16. I spoke with Deb Bryant, the conference chair, yesterday to get the low-down on the conference, and ended up getting a (polite) earful from some of the conference speakers.
I want to give GOSCON my full attention, because I think the Oregon State University/Open Source Lab-sponsored conference deserves a full mention all to its own, so I will defer describing this event (which I am pleased to be attending) until my next entry.
Until then, let me tell you about the discussion that ensued from some of the speakers about how the OpenDocument Format may not be the great unifier of documents after all.
What happened was this: during the call, Deb mentioned Gary Edwards, the co-founder and president of the OpenDocument Foundation, Inc., and asked had I heard of him? Sure, they were the ones that came up with the ODF for MS Office plug-in for Massachusetts last year (among other things). Buck "Marbux" Martin, Director of Legal Affairs for the Foundation, is also going to GOSCON, joining a panel discussion on Oct. 16 on Open Document Formats, along with Sun's Douglas W. Johnson, Ph.D., IBM's Arnaud Le Hors, and Microsoft's Jason Matusow.
Update: I got my conversation goofed up when I referred to someone from Groklaw being on this panel. Marbux is an active member of the Groklaw community, but he is not representing Groklaw in any way. My goof.
Deb wanted to ping me and Gary, to see about giving the panel a little extra attention, and when Gary replied, he included Marbux; William Welty CIO of the California Air Resources Board; and Andy Stein, the panel moderator and Director of Information Technology of the City of Newport News, Virginia. (I know Andy well; we met a few years back at LinuxWorld Boston.)
You all know how these joint e-mail intros are supposed to work: someone introduces you to someone else, you wait for a while to not appear too aggressive, then you say "hey," they say "hey," and eventually a real conversation might get going. Or not. Kind of like a middle school dance--only totally without the hormones.
Gary apparently didn't get that memo. No sooner than Deb e-mailed the him to intro me, I got a very lengthy how-do-you-do from Gary sent to the aforementioned group, which included one of the most interesting takes on the OOXML/ODF mess that I have read to date.
Being Mr. Reporter, I asked the tough, hard-hitting questions: "Can I re-print this?" (Ed Morrow, eat your heart out.)
The group allowed the re-print, so here is Gary Edward's take on OOXML and ODF:
Buck "Marbux" Martin will be representing the OpenDocument Foundation at GOSCON. He's far more a force for freedom and defending our open Internet future than i could ever be, as you will no doubt soon find out. I copied marbux on this reply.
This GOSCON Conference comes at an interesting moment in time. FOSS is banging hard on the interoperability barrier, and Microsoft refuses to budge. The European Union has had enough of this and is demanding Microsoft tear down the interoperability walls. The Bush Administration responds with their lassie-faire cloaked merchantilist position that competitors ought to pay Microsoft for access to the interop API's. Rampaging big vendor coalitions on both sides of the argument have trampled existing open standards processes and are now going at it tearing into each other. The commercialization of interoperability remains a key driver in both big vendor deals and big vendor standards consortia. FOSS is left on the outside looking in.
In the middle of this fray is the important argument over portable XML document formats. We believe that the conversion of existing documents, applications and processes to XML is one of the most important objectives facing governments, organizations and businesses.
The question is, which XML? ODF or MS-OOXML?
This question has a built in answer in that the problem statement by necessity references existing Microsoft documents, applications and processes since the MSOffice-Outlook desktop dominates workgroups the world over. The wonders of monopoly continue unabated.
Given the nature of day to day business processes, there is an important element to consider. This conversion to XML must be non disruptive. It must integrate into existing processes and infrastructure as a value added service and not as a new--disconnected but collaborative--process. The conversion process must fit into existing MSOffice-bound workgroup-workflow business processes. And that means a high fidelity "round trip" capable lossless conversion process.
The alternative to MSOffice-bound interoperability is that of rip out and replace. Which has been deemed by exhaustive pilot studies in Massachusetts, Denmark and Belgium to be costly, disruptive, and may in fact be "impossible" given the immaturity as developer platforms of the various MSOffice alternative solutions. (Excepting perhaps WordPerfect, now supporting ODF, MS-OOXML, and PDF).
The pragmatic approach the world must take is that of finding the means of being interoperable with the MS desktop. A difficult proposition if ever there was one.
The pressure is on in that Microsoft has been busy building out an entire stack of applications and services able to fully leverage their superior interop-integration with the MS desktop monopoly. The MS Stack crosses the grand convergence space of desktop, server, device and web systems. Every application is capable of speaking fluent MS-OOXML with highly proprietary and platform specific "Smart Tags" (replaces VBa scripts, macros, OLE, security as well as add a data binding/extraction model, metadata, forms, and workflow capability to document level objects).
The emerging MS Stack now includes MSOffice <> IE <> Exchange/SharePoint Developers Hub <> MS SQL-Server <> MS Dynamics <> MS Active Directory <> MS Collaboration-Media Server <> MSOffice Live <> MS Live.
Whew! Notice that in the MS Stack, MS-OOXML is the primary transport, the document/data container of interop-integration preference. There is no HTML!
We believe that ODF has never been the target of MS-OOXML. The target has been, since day one, HTML. With the grand prize the Internet and the age of collaborative computing.
The reason that ODF could not be the target ties into our initial XML objective statement. ODF was not designed as a response to the "big three" problems facing any transition of existing documents, applications and processes to XML. Those problems are:
- Compatibility with existing documents-file formats: including the volumes of MS binary documents.
- Interoperability with existing applications: including the over 500 million MSOffice-bound workgroups.
- Convergence of desktop, server, device, and web systems as fluid and highly interoperable routers of documents, data, and media. Also know as "Grand Convergence."
Most people reading such a list will ask why ODF was not designed to meet these needs? They are after all, imperatives of incredible consequence. If we can't convert existing documents, applications and processes to ODF, why bother with ODF?
Enter the "rip out and replace" crowd. These are ODF vendors and FOSS operatives who believe that the way to stop Microsoft is through anti trust inspired government mandates demanding the "rip out and replace" of MS Office. They insist that any compatibility compromise enabling ODF to solve the "big three" problems will diminish governments resolve to go forward with costly and disruptive "rip out and replace."
In this they are right. But with each passing day, we learn how badly this crowd over estimated that resolve. In truth, it looks like there wasn't much resolve to begin with. When ODF failed in Massachusetts, October 4th, 2006, the cornerstone of this approach was knocked into dust. So with this hope filled mistake, the "rip out and replace" crowd has led us down the rose garden path and into the clutches of disastrous briar patch.
On the other side, Microsoft is stalling, foot dragging, and doing whatever it takes to keep the desktop interoperability barriers in place until they can get past anti trust regulation and into the unimpeded roll out of the full MS Stack. To do this, Microsoft needs time. The stack is rolling out quietly, with Microsoft ever so quietly trying to migrate existing desktop bound business processes to the Exchange/SharePoint Hub where they are re written to embrace the MS-OOXML web. Still, they need to time to migrate and lock customer business into the E/S Hub, which sits at the core of the MS Stack. Once locked into a grand convergence stack of desktop, servers, devices and the web, it's too too painful to citizens and commerce for anti trust actions to unwind the damage. Shhhhh! Don't tell anyone but the E/S Hub is now at 65% marketshare. Shhhh! They've surpassed and supplanted Apache.
And one of the best ways of getting "time" is of course to muck up the ODF community and make certain that the "big three" problems are never addressed. Microsoft accomplished this we believe through deals with Sun and Novell.
So we have two ODF groups, each of which seeks to limit ODF interoperability and usefulness. The reasons are 180 degrees different, but limited ODF interop is nevertheless the result.
One group is comprised of Microsoft, Sun and Novell. Yes, the fox is in the ODF hen house. And has been since the 2004 Sun-Microsoft Interop-Patent-Hardware agreement. I would even argue that since 2002 with the inception of the OASIS ODF TC, and perhaps even with the open sourcing of OpenOffice, Sun used ODF and OpenOffice as threats to force Microsoft to the deal making table.
Novell joined in the Microsoft-Sun coalition in 2006, promising to write a MS-OOXML plug-in for OpenOffice! An effort that not surprisingly has since been joined by Sun. Now the dynamic duo is working on a native MS-OOXML version of OpenOffice!
The other "limit ODF Interop" group is that of IBM, Oracle and Google, with IBM clearly the driver. They of course seek to limit ODF Interop because they want a total rip out and replace of MS Office. Laudable goal, but proving to be unrealistic and unacceptable in the marketplace.
Interestingly, in the aftermath of the Novell - Microsoft deal, IBM seems to have finally figured out that Microsoft had them in a box. The recent increase of IBM influence and involvement in ODF and OpenOffice, some would call it a takeover, speaks to IBM's better late than never realization. But it also says something about Sun and Microsoft.
IMHO, Sun's release of their iron grip on both ODF and OpenOffice speaks volumes about Microsoft's readiness to release the full fury of the MS Stack. I mean, on one day we have IBM making this grand announcement, prancing around, basking in all their open source good citizen glory. And the next day we have Sun announcing that they are now a full fledged Windows OEM ready to cut the ribbon on their new Redmond Campus Interoperability Research Center.
Wow. Makes you wonder what it is that IBM paid for here? Looks like they just got thrown under the bus.
The IBM takeover, while great for Sun, is nevertheless a death blow to Novell. They have no choice but to fork OpenOffice. Sun's clearly the big winner here. They stalled and foot dragged on ODF, even when the Microsoft stalling and foot dragging came perilously close to blowing up the whole deal. Sun hung tough throughout. And it looks to me that with this recent announcement of MSOffice Live, most of the MS Stack pieces are in place. They need a SilverStream alternative to Glide to enable device level access to the MSOffice <MS-OOXML> E/S Hub <> MSOffice Live alignment, but they have plenty of time for that.
I would expect Adobe-Google-IBM to push OpenOffice into the grand convergence game now that they have Sun out of the way.
One last point on MS Office Live. It's about as non disruptive and process integrated a transition to collaborative computing as anyone could imagine. MS Office starts with existing processes and interfaces, with collaboration as a simple value added service. It's the one thing that Web 2.0, Office 2.0, Enterprise 2.0, SaaS alternatives can't touch. Interoperability with the MS Office-Outlook desktop.
So what about the OpenDocument Foundation? We fall into the middle area of trying to perfect the conversion to XML regardless of the fact that our two groups have the world caught between a rock and a hard place.
Like I said, interesting, in that regardless of how "right" people think ODF is over OOXML, it's still just one more thing for big vendors to fight about. In the end, Gary and the Foundation are saying, it's the customers that lose out, trying to get their documents opened.
It should be noted, however, that the OpenDocument Foundation is not just sitting around kvetching about this problem. They are working on a solution, which they promised they'd let me write about when it's officially ready.
The ODF/OOXML debate is about to get a lot more interesting.
The article says:
"The alternative to MSOffice-bound interoperability is that of rip out and replace. Which has been deemed by exhaustive pilot studies in Massachusetts, Denmark and Belgium to be costly, disruptive, and may in fact be "impossible" given the immaturity as developer platforms of the various MSOffice alternative solutions. (Excepting perhaps WordPerfect, now supporting ODF, MS-OOXML, and PDF)."
What a fascinating comment!
Despite the pluses and minuses of every other word processor on the planet WordPerfect remains the best of breed. No other word processor has the magnificent reveal codes feature. And if any company can do a really good job of inter-converting .wpd files (and thus .doc files as well) to .odt files then Corel is that company. And they may well -- single handedly -- give ODF more of a boost than any other conceivable company, whilst doing themselves a service at the same time.
If I read this article correctly, we're doomed, and with us the freedom to use anything other than Microsoft products. Someone please tell me how I'm wrong.
Two of the big three are true but the third is false. ODF will can interoperate between applications, I don't see why not, there are not technical reasons at least. Yes, replacing formats is costly, but is much more expensive in the long run, to be caught by M$ forever. Do the math.
Every country in the world except USA may be, is thinking in its long run convenience. If what the article say is true, M$ can stay with .doc format forever and dominate because is impossible to change the current used formats. Doesn't need OOXML get involved.
The true is, governments are looking for an excuse to dump M$, using the argument "it's not standard" (a really good argument), and M$ got the message and try to put OOXML as standard too at any cost (but no luck until now).
Of course M$ see "OOXML standard" as a new propietary .doc format, difficult to implement in other platforms, with many patented code, and of course incomplete specification. In this way all will change to stay the same.
"Yes, replacing formats is costly, but is much more expensive in the long run, to be caught by M$ forever. Do the math."
Ah, but you're forgetting something. The average CEO is only interested in the next 3-5 years, tops, so they can cash in those stock options. They really don't care about accessing documents 20-50 years down the track.
ODF v1.2 WILL GIVE YOU 100% PERFECT FIDELITY WITH LEGACY MICROSOFT OFFICE DOCUMENTS TO THE SAME EXTENT AS MICROSOFT CLAIMS FOR OOXML.
http://opendocument.foundation.googlepages.com/home
The only problem (as with OOXML implementations) is that documents using Microsoft secret and undocumented formats and behaviours can only be rendered by applications produced by Microsoft applications or those produced by Microsoft licensees under NDA and patent licenses from Microsoft. This basically means that ONLY MICROSOFT APPLICATIONS OF THOSE OF MICROSOFT LICENSEES CAN ATTAIN 100% COMPATIBILITY WITH MICROSOFT LEGACY FILES whether ODF or OOXML is used. We can only hope that the EU anti-trust authorities prosecute and fine Microsoft to force it to reveal it's secret formats and behaviours, so that true interoperability with legacy documents is possible.
Having said this, ODF + DaVinci Plug-in on Microsoft Office is vastly preferable to OOXML, because in addition to the secret and undocumented formats and behaviours required for full compliance with OOXML, there are the following issues:
1) ODF 1.2 can handle other legacy formats in the same manner (like Wordperfect) - it not simply limited to Microsoft legacy formats as OOXML is.
2) Microsoft has said it won't comply with the OOXML spec it has submitted for ISO approval anyway, so what is the purpose of OOXML?
3) Microsoft's patent covenent excludes future versions or modifications to OOXML, hence OOXML won't be compatible with MS Office formats in future, and because unlike the current binary formats, which can be reverse engineered, it will not be possible to legally reverse engineer Microsoft's future variants of OOXML. Hence OOXML will be more closed and more proprietary than any previous Microsoft file format.
4) The OOXML refers to things not described in the OOXML specification that require licensing of Microsoft patents which because they are not described in the OOXML specification itself, but elsewhere, are not covered by Microsoft's covenant not to sue. Hence full compliance with the OOXML spec is only possible for Microsoft or Microsoft licensees.
We are doomed, or so it seems.
I don't see people worried about document format freedom, they just don't care. They don't even bother to tryout other word applications. Thats true for everyone but except a handful of geeks on a few companies (me included).
People's documents interoperability is a must, so everyone will always have to be MS-compatible, whether it is .doc or OOXML doesn't matter much. Why convert to ODF or whatever when you can use "the real thing"?
I don't see a solution. We will continue to chew up whatever MS throws at us. I just hope MS alternatives will get so good that you won't care if the rest of the non-geek world uses something different.
Doomed, NOT!
If there was ever a rallying cry that could inspire Free and Opens Source developers to pull together, and support one another, this is it. Where on this planet could M$, or any other monopolist, muster the resources to create interoperability between ODF supporting systems? Only the Free / Open Source communities can address this shopping list of functionality with timely alternatives.
Seriously!
To reply, remove M$ modus operendi from addie.
Actually the old binary Microsoft formats aren't as much of an interoperability problem as OOXML. OpenOffice will for example read it perfectly well in 98% of cases. Just use ODF and wrap the legacy binaries in XML. OpenOffice and other office suites will read it as well as they do now. The most important thing is not to adopt the new proprietary format OOXML which will also make new documents proprietary (by virtue of Microsoft's patented and secret formats and behaviors). Wrapping the legacy binaries in XML or just keep the legacy binaries in legacy binary format. The thing is that if you have lost content information due to the proprietary nature of Microsoft formats, you aren't going to save it by putting it into another proprietary Microsoft format. What you can do is to keep any new data getting into a proprietary format.
This article says:
"If we can't convert existing documents, applications and processes to ODF, why bother with ODF?"
This is utter nonsense. Just use OpenOffice to open old MS documents and save them as ODF. It is very simple. OpenOffice is even better at opening old MS Formats than MS Office itself. I have done it many times. Try it yourself.
I guess the Linux movement is about to pack up their code and Internet connections and stay home alone.
What I see are many companies that are going to hurt in the future. You have the smart ones finding Linux and then you have those that will wallow for a bit and then die in their own filth, unable to move, overburdened with costs, and stuck crying to Redmond to help them meet all of their needs to remain competitive. As they die off or take the plunge and migrate away....
Regardless of what major vendors decide to do, I will continue to push for Linux growth at the desktop level. You can't beat the Linux proposition and Microsoft is clearly feeling the heat. They are extremely leveraged (all or nothing) and they will eventually get blown out of the kitchen.
Hi, Brian,
Still further clartification needed. I have not been active in the Groklaw community for many months.
Best regards,
Marbux
"So what about the OpenDocument Foundation? We fall into the middle area of trying to perfect the conversion to XML regardless of the fact that our two groups have the world caught between a rock and a hard place."
Their magic tool which, according to the brochure, takes advantage of a transient version of the Word document in memory (probably a variant of RTF), will at best work only for Word documents.
How can Gary Edwards and al. claim to have a better interoperability tool for MS Office documents when their magic tool does not address spreadsheets or presentations?
Who does Gary Edwards actually work for?
At the moment Sharepoint is awful. Technically it's behind where Lotus Notes was back in the mid 90s! It will come on but that takes time. People seem to buy it because it's Microsoft and there's this magic aura in business to anything they do.
We've seen that Microsoft only invest in development of their products when they fear competition. Just look at Internet Explorer's history. If they could move to a software rental model with complete lock-in then they'd never have to innovate or develop again - and I can see this happening. It would be the ideal dream for those in charge but a terrible death for computing.
It is this risk that I believe must be realised and must be stopped.