Is VML in or out now, or was that a typo?

Its already June the 22nd, and how time flies. In Malaysia, the "suspended" TC4 has only till June 30th to submit comments on the Ecma Office Open XML draft specification to SIRIM.
I decided to get going by making sure I have the correct documents. On downloading, I was just checking the files, and I came across this rather strange page. I looked at it, and thought that there may be some issues with my PDF viewer.

So I downloaded the official viewer from Adobe just to make sure it was not because of my computer.

This is what I saw:

Look at the arrows. The "fillcolor=#..." does not have a trailing closing quote. Instead it has a strange code "[3204]". I tried looking for this information, but its in another Part (not downloaded yet). Im not sure if "[3204]" is a valid attribute in that XML fragment. Then again, there is a missing trailing quote for "fill darken". Instead it has yet another strange number "(118)".

You can see this for yourself in Part3 Section 2.3.1, Page 4 (as on the page, or page 15 if you search via the PDF viewer).

I just hope that these are typos, otherwise it would certainly look like really weird XML. If they are typos, I wonder how many there are in the spec? Surely the guys at Ecma had enough time to review this? Why am I now suppose to act as a junior sub editor? I would understand if it was a missing quote or full-stop here or there. But three quite major glitches within consecutive 3 lines? Thats quite scary.

I then scrolled up to see what this section was all about. What I read next surprised me to no end.

Right there, it simply states:

"All background information in a WordprocessingML document is stored using the Vector Markup Language (VML) syntax. The single exception to this is the background color, which is stored natively in WordprocessingML using the bgColor attribute."

If you use plain colours on your documents then you can simply use WordProcessingML. Otherwise, if you want fancy gradiated backgrounds, you use VML. There is nothing wrong with VML,
EXCEPT THAT IT IS DEPRECATED IN MSOOXML!

In Part 1, Section 8.6.2 Page 15 (27), it clearly states:

[Note: The VML format is a legacy format originally introduced with Office 2000 and is included and fully defined in this Standard for backwards compatibility reasons. The DrawingML format is a newer and richer format created with the goal of eventually replacing any uses of VML in the Office Open ML formats. VML should be considered a deprecated format included in Office Open XML for legacy reasons only and new applications that need a file format for drawings are strongly encouraged to use preferentially DrawingML. end note]

This note appears many places throughout the MSOOXML spec, so it must be really important.

So:

1. VML is included for backwards compatibility reasons only.
2. Drawing ML is newer and richer and designed to replace VML.
3. VML is deprecated and only included for legacy reasons
4. New applications are strongly encouraged to use DrawingML instead

This simply begs these questions:

1. Why then is Section 2.3.1 explicitly using VML?
2. Where are the instructions for using DrawingML instead?
3. If DrawingML is richer and designed to replace VML, why have VML in the first place?
4. I would take it that Office 2007 is a "new application" and should use DrawingML instead of VML. Does it?

Ive just reviewed 2 pages of this specs, and my head already hurts.

Please help?!

[Coincidentally:
In my discussions about failed standards (like VML) piggy backing MSOOXML to international standardisation-hood,  a blog reader  Jeffrey commented:

"That seems you distorting the the truth by a mile. VML is added as deprecated for support of converted MS Office XML 2003 formats. That means it is for backwards compativility and not for rushing it trough as a standared."

To which a Mr Stephane Rodriguez replied Jeffrey:

"It seems it's you [Jeffrey] who are distording the facts, or the specs is simply wrong.

If you think VML is deprecate, then please follow those steps :

1. create a new Excel 2007 spreadsheet ;
2. right-click on a cell and choose "Insert comment" ;
3. type a comment ;
4. save the spreadsheet ;
5. close it.
6. Now unzip it.
7. What do you see? A nice VML part.

There you go with your deprecate part."

There you go indeed. I couldn't find the citing for this "SpreadsheetML Comment" section, because it would take me ages just to comb through the chaff. But at least now we have a concrete citing at Part 3, Section 2.3.1 which clearly states that VML is considered non legacy nor deprecated. ]

yk.

Comments

[Stephane Rodriguez]

To answer your question about SpreadsheetML VML, it's referenced in section 6.4 (page 4920), and is pretty much a copy/paste of section 8.6.2 that you have reprinted above.

But in fact, there is no specs-wide list of where VML appears, especially in NEW files. For instance, VML is also used if you insert a new OLE object.

There is a reason why Microsoft does not document this stuff. They would have to give the documentation of OLE, which in turn would show clearly as in water that this stuff is platform-dependent : Windows.

That Microsoft goes as far as not even listing where VML is used, is also probably because it brings back a taboo subject. Which I'll address right now.

Back in 1998, Mr Gates and his crownies had bet on WYSIWYG HTML. That's how they convinced themselves that they had finally taken the internet train. Of course, when Windows 98 shipped, it was with Internet Explorer bundled with the operating system. So with that in hands, Microsoft planned to make sure that this OS + browser coupling would never end. To that end, they made sure the Office team would create a dependency on it. Office already had created a dependency on Windows (binary formats, OLE, ...) but apparently making a productivity app depend on an OS was not enough for them, they chose to also make sure that Office would become a recruitment tool for Internet Explorer, and vice versa.

Hence, one of the BillG's memo whose copy you'll find here : http://antitrust.slated.org/www.iowaconsumercase.org/011607/2000/PX02991.pdf

As amazing as it reads by itself, with BillG making it very clear that Office implementing open standards was a mistake and that it would cause the death of Windows (i.e. the dependency as mentioned above), he mentions what we might want to call a new trojan horse, using Internet Explorer to view Office documents (Word, Excel, Powerpoint) with only a web browser. In other words the WYSIWYG HTML thing.

How does this memo relate to the discussion about VML?
Well, it's very simple, in the entire history of the Office product group, only once did they work with the Internet Explorer team. And that was to implant what is now known as the VML library. The VML library is surfaced as a proprietary markup whose semantics is not documented and is a direct XML serialization of the shared drawing library in Office, internally known as MSO (codename Escher).

So the use of VML inside Internet Explorer was the trojan horse which would ensure that Office documents shared on the web would only render on Windows. There is no VML implementation out there other than Internet Explorer's. And again, there is Microsoft IP in it, so that's probably a reason why.

That was the days of Office 2000, XP, 2003 and 2007. In Office 2007, the Office team wanted to move to another markup language for their graphics layer. Not because of all a sudden, they wanted to remove a dependency on the OS and Internet Explorer, it's because they wanted to add new features and create yet another dependency. In Office 2007, DrawingML is used to describe a couple of 3D effects such as bevel and so on. The semantics and actual layout algorithms are not documented at all. They are extensively used in the new chart drawing engine for instance. And, as a result, since the old chart drawing engine, and the old shared MSO are gone with it, it means that opening an existing file that contains a chart will be rendered differently in Office 2007 than in older Office versions. For instance, if you create an Excel 2000 file with a chart, and open that file in Excel 2007, you'll see something different. So much for "100% backwards compatibility"...

Last but not least, since the Office team could not, or did not want to, make a full move to DrawingML in Office 2007, there are bits of VML all over the place, be it in NEW Word 2007 files, NEW Excel 2007 files and NEW Powerpoint 2007 files. This comes in contradiction with the specs proposed for ISO standard, and in practice this also puts a great barrier to entry : it requires a consuming application to support VML if said application wants to render it.
Of course, said application in some cases may simply ignore the VML parts, but then if your application is supposed to render for example an Excel spreadsheet containing a comment on it, and ignores VML, then customers will see it as a bug! and it turns a black eye to a non-Microsoft application in favor of Microsoft Office applications. This technique is nothing other than lock-in. They are making sure the barrier to entry is very high for applications willing to be rich consumers of those files, and in practice it means Office itself is always better than anybody else. Again, this contradicts the specs. An international standard is supposed to provide fair and equal footing to everyone implementing it.

So the Office team at Microsoft could not manage to get rid of VML. It is possible, unless they intentionally use it as a lock-in tool, that next version of Office, codenamed Office 14, will be the first one to have an Office-wide DrawingML implementation. Until then, Office 2007 is at odds with the specs that Microsoft is pushing and as a result, it is not a reference implementation of the specs!
In other words, since there cannot be a non-Microsoft implementation of the specs (it will take at least ten years for a company starting now, and it will never achieve 100% fidelity since many things are missing in the specs), it follows that there is simply NO reference implementation out there! Quite amazing...

One only hopes that, in the meantime Microsoft won't change their mind again, and that they not only will keep VML for the foreseeable future, but also will create a new dependency to a Windows-only API. The new graphics .NET 3.0 based API called Windows Presentation Foundation comes to mind, obviously...

We'll see.

[anonymous]

In MS-ECMA-OOXML Part 4 "Mark-up language reference" there are a lot of references to "DiagramML". Example:

5.9.3.5 pt (Point)
This element defines a point in DiagramML. A point in DiagramML is defined to hold data associated with a particular point or node in a diagram."

I didn't hear about it. It is just another internal format of Microsoft to be super-fast-tracked by ISO together with DrawingML, MathML, VML, OPC, etc, etc?

[anonymous]

"Word was only able to do the work to upgrade to the new DrawingML framework for Charts; Smart Art; and Pictures. All other drawings still use the legacy VML architecture. That's why we needed to include both VML and DrawingML in the standard; otherwise folks wouldn't have been able to fully interop. The hope of course is to move to use DrawingML for everything in future versions, as Excel and PowerPoint did."

http://blogs.msdn.com/brian_jones/archive/2006/10/09/Office-Open-XML-final-draft_210021002100_.aspx

[Peter]

VML get back with a confrontation with SVG and stand out fine on that website:
http://www.svg-vml.net/