Subject: Re: Registration of media type application/calendar+xml

From: Keith Moore <[email-masked]>
Date: Thu, 9 Sep 2010 21:44:07 -0400

This was a bad idea when it was first proposed (if I recall correctly) around ten years ago, and it's still a bad idea.

Whenever you define an alternate representation of something, there will inevitably be skew between the original representation and the alternate representation.

This remains true even if you define mapping rules between the two (as you do in this document).   The problem with mapping rules, which I believe I pointed out around ten years ago, is that the rules are specific to a particular version of iCalendar.  If either iCalendar is extended, or the XML representation is extended, there's no guidance as to how to map the extended format into the other representation.

In addition, defining a new calendar format harms interoperability even if you can keep the two representations in sync.  The reason is that it's no longer sufficient for a calendar application to support just one representation of calendar data.  In order to reliably interoperate, it must at least able to read both, and it probably needs to be able to write both.  That, and when sending calendar data to other applications, either both representations must be sent, or some way of negotiating which format to use is needed, or the user must be asked to choose which format to export.

In summary, this is a thoroughly bad idea which can only do harm.

Please withdraw this proposal, or at least withdraw the types application until the proposal has enjoyed more review.

Keith

On Sep 2, 2010, at 6:44 PM, Steven Lees wrote:

> To:
> [email-masked]
> Subject:
> Registration of media type application/calendar+xml
> Type name:
> application
> Subtype name:
> calendar+xml
> Required parameters:
> none
> Optional parameters:
> charset, method, component and optinfo as defined for the text/calendar media type
> Encoding considerations:
> iCalendar data is typically UTF-8 and thus the XML representation will follow that. As a result, for 7-bit transports, data in UTF-8 MUST be encoded in quoted-printable or base64.
> Security considerations:
> This extension does not introduce any new security concerns than those already described in iCalendar (RFC5545).
> Interoperability considerations:
> This media type provides an alternative syntax to iCalendar data based on XML.
> Published specification:
> This specification. (http://www.ietf.org/id/draft-daboo-et-al-icalendar-in-xml-06.txt)
> Applications which use this media type:
> Applications that currently make use of the text/calendar media type can use this as an alternative.
> Additional information:
> Magic number(s): None
> File extension(s): XML data should use "xml" as the file extension.
> Macintosh file type code(s): None specified.
> Person & email address to contact for further information:
> Steven Lees
> EMail: [email-masked]
> Intended usage:
> COMMON
> Restrictions on usage:
> There are no restrictions on where this media type can be used.
> Author:
> Cyrus Daboo
> EMail:  [email-masked]
> Mike Douglass
> EMail:  [email-masked]
> Steven Lees
> EMail:  [email-masked]
> Change controller:
> IETF
>

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 09 Sep 2010 22:22:28 -0400

Hi Keith,
(cc'ing Apps area ADs - I believe Peter intends to "sponsor" this draft so 
he at least should be aware of our discussion here)

--On September 9, 2010 9:44:07 PM -0400 Keith Moore <[email-masked]> 
wrote:

> This was a bad idea when it was first proposed (if I recall correctly)
> around ten years ago, and it's still a bad idea.

An XML representation for iCalendar is vital if we are to keep iCalendar 
relevant in the web-based world. The drive for this work comes from a 
number of areas - in particular the smart grid effort sponsored by NIST 
will make use of this as part of the standards suite they are defining.

> Whenever you define an alternate representation of something, there will
> inevitably be skew between the original representation and the alternate
> representation.

iCalendar is itself extensible in terms of the addition of new properties 
and parameters. The trivial mapping of the iCalendar object model to our 
XML schema allows such extensions to be trivially mapped.

> This remains true even if you define mapping rules between the two (as
> you do in this document).   The problem with mapping rules, which I
> believe I pointed out around ten years ago, is that the rules are
> specific to a particular version of iCalendar.  If either iCalendar is
> extended, or the XML representation is extended, there's no guidance as
> to how to map the extended format into the other representation.

If iCalendar is extended in such a way that it is no longer compatible with 
the XML mapping, then it would really be a completely new version that 
would itself not be compatible with the existing iCalendar v2.0. If that 
happens, well compatibility has clearly been thrown out the window.

> In addition, defining a new calendar format harms interoperability even
> if you can keep the two representations in sync.  The reason is that it's
> no longer sufficient for a calendar application to support just one
> representation of calendar data.  In order to reliably interoperate, it
> must at least able to read both, and it probably needs to be able to
> write both.  That, and when sending calendar data to other applications,
> either both representations must be sent, or some way of negotiating
> which format to use is needed, or the user must be asked to choose which
> format to export.

Correct - having two representations can cause problems like this. However, 
with the advent of HTTP-based calendar protocols, the content negotiation 
mechanisms in HTTP (e.g. Accept header) can be used to push most of the 
burden for this on servers rather than clients.

> Please withdraw this proposal, or at least withdraw the types application
> until the proposal has enjoyed more review.

Having a MIME type for this representation is critical to enabling content 
negotiation features. I feel strongly that this is a valid and useful 
proposal and we are already seeing uptake in the form of smart grid work 
and others.

Whilst this is an individual submission, I would point you to 
<https://datatracker.ietf.org/doc/draft-ietf-vcarddav-vcardxml/> which is 
the product of the vCardDAV working group. That specification defines a 
mapping of vCard to XML which follows the same basic design of our 
iCalendar-in-XML work.

-- 
Cyrus Daboo

---

From: Keith Moore <[email-masked]>
Date: Thu, 9 Sep 2010 22:51:10 -0400

On Sep 9, 2010, at 10:22 PM, Cyrus Daboo wrote:

> Hi Keith,
> (cc'ing Apps area ADs - I believe Peter intends to "sponsor" this draft so he at least should be aware of our discussion here)
> 
> --On September 9, 2010 9:44:07 PM -0400 Keith Moore <[email-masked]> wrote:
> 
> An XML representation for iCalendar is vital if we are to keep iCalendar relevant in the web-based world. The drive for this work comes from a number of areas - in particular the smart grid effort sponsored by NIST will make use of this as part of the standards suite they are defining.

Somebody needs to talk some sense into those people.  Defining another calendar format will only harm interoperability.  It doesn't save any calendar implementation from needing to implement another parser, because if it wants to interoperate with existing products or be able to read old events it's still going to have to support iCalendar and probably vCalendar also.  So what's the _technical_ (not political) benefit from doing this?

IETF's job is to provide technical leadership, not to follow bad advice.  Instead of caving into them, what we need to do is publish a draft called "Translating Everything into XML Considered Harmful".

> iCalendar is itself extensible in terms of the addition of new properties and parameters. The trivial mapping of the iCalendar object model to our XML schema allows such extensions to be trivially mapped.

But your document doesn't define what the mapping is.  And your mapping spec (not to mention implementations) will always lag extensions.

> If iCalendar is extended in such a way that it is no longer compatible with the XML mapping, then it would really be a completely new version that would itself not be compatible with the existing iCalendar v2.0. If that happens, well compatibility has clearly been thrown out the window.

Sure, but if that should actually be needed, why do you want to make it even more difficult?    If and when there were a need for an iCalendar 3.0 - i.e. an incompatible change because there was no way to shoehorn the new features into the old format - it might make sense to define _that_ to use XML.

> Correct - having two representations can cause problems like this. However, with the advent of HTTP-based calendar protocols, the content negotiation mechanisms in HTTP (e.g. Accept header) can be used to push most of the burden for this on servers rather than clients.

Provided, of course, that HTTP-based calendar protocols are being used.   But currently, all of the iCalendar content I receive is over email.

Even if you have content-negotiation, having two formats still means there are two code paths, which introduces additional opportunities for failure.

> Having a MIME type for this representation is critical to enabling content negotiation features.

Let me be clear.  I'm not saying just withdraw the MIME type application.  I'm asking you to abandon the internet-draft also.  Don't introduce an incompatible iCalendar format.  Then you don't need content-negotiation for calendar data.

> I feel strongly that this is a valid and useful proposal and we are already seeing uptake in the form of smart grid work and others.

I don't doubt at all that there are people jumping on the bandwagon.  That's not a sound justification for degrading interoperability of calendar products.

> Whilst this is an individual submission, I would point you to <https://datatracker.ietf.org/doc/draft-ietf-vcarddav-vcardxml/> which is the product of the vCardDAV working group. That specification defines a mapping of vCard to XML which follows the same basic design of our iCalendar-in-XML work.

One example of bad practice does not justify another.  That is what is known as propagation of error.

Keith

---

From: [email-masked]
Date: Thu, 09 Sep 2010 20:38:49 -0700 (PDT)

> This was a bad idea when it was first proposed (if I recall correctly) around
> ten years ago, and it's still a bad idea.

I strongly disagree.

> inevitably be skew between the original representation and the alternate

This is demonstrably false. The Sieve in XML representation specified in RFC
5784 provides a counterexample - the way the format and mapping is defined,
there's no way to represent anything in the XML variant that cannot be
represent in the regular variant, and vice versa.

In the case of Sieve, introducing the sort of skew you're worried about could
only be done by violating RFC 5228 on the regular format side (by changing the
core language syntax) or RFC 5784 on the XML side (by introducing an
incompatible schema change).

I have only done a cursory review of iCalendar in XML specification, but I
believe it covers this issue adequately, and according to the document, it
clearly intends to define a format that fully support round trips between the
representations. If you believe it does not, it is up to you to provide
specifics of how it does not, rather than asserting there's a problem without
any actual evidence.

> This remains true even if you define mapping rules between the two (as you do
> in this document).   The problem with mapping rules, which I believe I pointed
> out around ten years ago, is that the rules are specific to a particular
> version of iCalendar.  If either iCalendar is extended, or the XML
> representation is extended, there's no guidance as to how to map the extended
> format into the other representation.

Also demonstrably false, and once again RFC 5784 provides a counterexample.
When Sieve extensions are defined the only thing that ever needs to be updated
in the mapping to accomodate them is the list of controls, which is needed to
map to the XML format. (And out of the many Sieve extensions that have been
specified, exactly one introduced new controls, and additional extensions of
this sort are very unlikely.) Morever, had this one list been viewed as a
problem it could easily been eliminated, at the cost of making the XML
representation a little less clear. 

Again, if you believe that introduction of new iCalendar properties will
require updating the mapping to an unaceptable degree, it is up to you to cite
specific exmaples instead of just asserting there's a problem.

> In addition, defining a new calendar format harms interoperability even if
> you can keep the two representations in sync.  The reason is that it's no
> longer sufficient for a calendar application to support just one representation
> of calendar data.  In order to reliably interoperate, it must at least able to
> read both, and it probably needs to be able to write both.  That, and when
> sending calendar data to other applications, either both representations must
> be sent, or some way of negotiating which format to use is needed, or the user
> must be asked to choose which format to export.

Yes, the existence of multiple formats can be an issue, but so can the
inability to easily manipulate application data in popular environments using
readily available tools. IMO the tradeoffs in this case are *overwhelmingly* on
the side of being able to manipulate calendar data using XML tools.

It should also be noted that in the case of XML, the existence of certain of
those tools also lessens the difficulty of implementing conversions from XML to
the original format. I know I sound like a broken record, but once again  RFC
5784 provides an example of this: It includes a stylesheet to convert from the
XML representation to the original format.

FWIW, I think including such a conversion stylesheet would be very useful.

> In summary, this is a thoroughly bad idea which can only do harm.

Again, I strongly disagree.

> until the proposal has enjoyed more review.

First of all, I'm actually kicking myself now that Sai and I didn't include a
registration for application/sieve+xml in RFC 5784. If we ever do a revision
I'll be sure to correct that omission.

Second and far more important, the ietf-types list is for preliminary reviews
only. In the case of a standards tree type like this that's being defined in an
RFC, approval comes from the IESG, and that will happen as part of the review
and approval process for the actual specification. It is therefore entirely
appropriate for this registration to be posted to the list at this time so it
can be reviewed.

Now, on to the registration itself...

> > To: [email-masked]
> > Subject: Registration of media type application/calendar+xml
> > Type name: application
> > Subtype name: calendar+xml
> > Required parameters: none
> > Optional parameters:

> > charset, method, component and optinfo as defined for the text/calendar

I have to question the inclusion of the charset parameter here. XML has its own
internal charset label, and having this parameter opens the door to charset
label silly states. Perhaps it would be better to map the text/calendar charset
parameter to the internal XML label, and vide versa.

> > iCalendar data is typically UTF-8 and thus the XML representation will
> > follow that. As a result, for 7-bit transports, data in UTF-8 MUST be encoded
> > in quoted-printable or base64.

It is customary to include a value of 7bit, 8bit, or binary here and I believe
8bit is the correct value. While it is true that XML can use UTF-16 and similar
charsets that require a binary encoding label, this is a mapping of a text
type, which cannot employ UTF-16.

> > This extension does not introduce any new security concerns than those
> > already described in iCalendar (RFC5545).

The use of XML actually raises a few additional security considerations that
should be mentioned. The text in section 5 of RFC 5784 covers these.

> > File extension(s): XML data should use "xml" as the file extension.

This is entirely your choice, but I'm not sure this recommendation is a great
idea.

> > Person & email address to contact for further information:  Steven Lees

Although this is an individual submission, it's in the standards tree so you
might consider referring this to the calendar discussion list.

> > Change controller: IETF

				Ned

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 01:09:09 -0400

On Sep 10, 2010, at 12:34 AM, Phillip Hallam-Baker wrote:

> The objections raised by Keith do not appear to me to fall under any of the requirements for MIME type registration set out in RFC 4288.

I didn't claim that they did.  I'm just taking the opportunity to ask that the proposal be voluntarily withdrawn.

> I disagree with the argument made in any case.
> 
> If you want to have a system in which 95% of your data structures are in XML you probably don't want to have to introduce a separate syntax and you most certainly do not want to deal with a separate data model for dealing with calendaring.

It's not at all clear that trying to coerce 95% of the data models in your system to be compatible with XML is a worthwhile goal.  The XML data model is tortured.   Trying to impose it on network protocols should be regarded as an act of violence.

At any rate, this proposal doesn't change the iCalendar data model - it just makes it harder to use.  If you have a beef with the iCalendar data model, feel free to try to come up with a better one.

> The iCalendar format represents a 1990s style approach to the problem. There is no real separation of syntax from the data model. Software developed in that manner is notoriously difficult to get right for the reasons that Keith describes. 

XML has lots of problems of its own.  I recently had to review a specification that referenced WS-i and WS-security and about a couple of thousand other pages of useless crap that went with it.  All for the sake of being able to transmit about six meaningful name-value pairs from a client to a server.  It was completely ridiculous.

I'm no fan of the iCalendar format, nor the vCard and vCalendar formats that preceded it.  But for all of its purported generality (and perhaps because of it) XML has turned out to be no better, and in many ways worse.  It's amazing how hard people will work to make a simple idea complex, especially when the simple idea has become a bandwagon, or (in this case) a religion.  

In principle, it would make sense for things to have a uniform syntax.  But XML gets this wrong in several different ways.  The most obvious is that XML data structures don't map well onto data structures supported by programming languages.   That's probably because SGML wasn't designed to do that - it was designed to mark up text.  Another problem is that XML confuses typing and naming.   Another problem (especially when mapping other structures to/from XML) is that the distinction between parameters and sub-elements is pretty much arbitrary.

> XML is a substantial overhead if you are dealing with a single protocol but when you are dealing with multiple protocols the benefits are substantial and allow something like 70% of your coding effort to be pushed onto the platform layer. That means that you have 70% less new code and new code paths to contend with.

If your programmer is spending 70% of his coding time dealing with a presentation layer, even one as convoluted as iCalendar, you should fire him.  It's not like regular expression parsers are hard to come by these days.   Nor are libraries that can parse standard formats hard to come by.

Another of the big problems with the XML religion is that its adherents have the mistaken impression that defining the syntax is most of the work of defining a protocol - so that once you decide to use XML, most of the work is done.  Apparently, semantics don't matter much.

Another problem with XML is that it makes data models so easy to extend (just add more element definitions) that people often don't take due care in defining their data models.

> One of the discoveries of the mid 1990s was that yacc and LR(1) grammars are no more useful for describing computer languages than they are for describing natural languages.

That's a ridiculous statement.  (Okay, maybe strictly speaking LR(1) isn't quite enough, but it's close enough for most computer languages that you can usually make an LR(1) parser work.)  Computer languages don't need the same kind of expressive power that natural languages have.  Natural languages have to allow for a certain amount of ambiguity, but that's a liability in computer languages.

> The most useful feature of a computer grammar is regularity and consistency. XML enforces a high degree of consistency.

It enforces consistency in syntax. Taken by itself, that's a good thing.  But when you parse XML you don't get a very usable data structure.  You get a mess.  And once you do the work of transforming that data structure into an effective internal representation, you've negated whatever advantage you might have found in not having to have written a parser/generator for it.  You haven't solved the problem of needing a parser and generator - you've just moved it.  Instead of parsing text you're parsing a DOM structure.  You've added an extra layer or two for no benefit.

You're essentially arguing the syntax by which data is represented on the wire - the presentation layer - should constrain how data is represented internally in a system.   And then you're arguing that the particular constraints imposed by XML are appropriate constraints.    That's brain damage.

> Now I would quite prefer to take about 50% or more of the XML spec and discard it. They did a good job of taking out the most insane features of SGML but there is much more cruft that could be cut out. But that does not change the fact that using XML as is produces clearer specifications that are more likely to be implemented without errors than with the 1990s approach.
> 
As is often the case, you're simply and utterly incorrect.

Let's stamp out XML in our lifetime.  Even FORTRAN deserves to live longer.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 01:40:16 -0400

> represent in the regular variant, and vice versa.

RFC 5784 might well be a valiant effort toward that.  But it has only been out a few months, so I think the jury is still out as to how well it works in practice.

I also think it helps that RFC 5784 specifically says it's not intended as an alternate storage format for Sieve.  I can certainly see some utility in being able to use XML tools to manipulate things not originally written in XML.

But here's the acid test.  If you can define a mapping from iCalendar to XML that doesn't require any string constants to describe it (other than for iCalendar keywords that imply nesting, and separators that are used in a regular fashion in iCalendar), and if you can define the inverse mapping from XML to iCalendar without naming more than a couple of specific element or parameter names - then I'll concede that the mapping will probably continue to work in the face of extensions to the iCalendar data model.   Otherwise, I'm highly dubious. 

Even then, this would not address the degraded interoperability resulting from the need to have multiple formats.

> specifics of how it does not, rather than asserting there's a problem without

I disagree, because there are interoperability problems resulting from the introduction of an alternate format even if the mappings remain invertable in the presence of extensions.

> map to the XML format.

I think you've just illustrated my point.

Though really I'm not as worried about Sieve as I doubt that there's nearly as much deployment of Sieve as there is of iCalendar.    I think it's far more likely that all Sieve implementations can be upgraded to support XML, if there's a desire to do that, than that all iCalendar implementations can be upgraded to support XML.  

I also think that Sieve<>XML is inherently saner than iCalendar<>XML because Sieve (being essentially a programming language) is already structured similarly to the way XML is structured.

> specific exmaples instead of just asserting there's a problem.

I think that the burden is on the proposers to justify producing an incompatible alternative to a existing standard, which will definitely harm interoperability.

> the side of being able to manipulate calendar data using XML tools.

If people want to define mappings between iCalendar and XML just for the sake of being able to manipulate the data using XML tools, that doesn't bother me so much.   It's when people want to start exchanging calendar data using XML in some cases and iCalendar in others that the interoperability problems start cropping up.  (Though I have to wonder how much it really helps to be able to manipulate calendar data using XML tools, as calendar manipulations tend to be fairly specialized to calendar applications.)

> appropriate for this registration to be posted to the list at this time so it

Yeah, I know...that's why I cc'ed the IETF list.   I am not arguing that the type registration itself is improper; I'm arguing that defining a new representation of iCalendar is a bad idea and should be discouraged by the IETF community.  And for that reason I'm asking the document authors to put the type registration on hold.

Keith

---

From: Tony Finch <[email-masked]>
Date: Fri, 10 Sep 2010 08:41:05 +0100

On 10 Sep 2010, at 06:09, Keith Moore <[email-masked]> wrote:
> 
> If you have a beef with the iCalendar data model, feel free to try to come up with a better one.

Funny you should say that :-)
http://fanf.livejournal.com/104586.html

Tony.
--
f.anthony.n.finch  <[email-masked]>  http://dotat.at/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 10 Sep 2010 09:50:35 +0200

On 10.09.2010 04:51, Keith Moore wrote:
> ...
> ...

For me, the opposite is true.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 10 Sep 2010 12:11:27 +0200

On 10.09.2010 07:09, Keith Moore wrote:
>> If you want to have a system in which 95% of your data structures are
>> in XML you probably don't want to have to introduce a separate syntax
>> and you most certainly do not want to deal with a separate data model
>> for dealing with calendaring.
>
> It's not at all clear that trying to coerce 95% of the data models in
> your system to be compatible with XML is a worthwhile goal. The XML data
> model is tortured. Trying to impose it on network protocols should be
> regarded as an act of violence.
> ...

That's *far* from clear. Could you elaborate in the context of 
calendaring? As far as I can tell, the torture is more about having to 
parse what we have today.

> At any rate, this proposal doesn't change the iCalendar data model - it
> just makes it harder to use. If you have a beef with the iCalendar data
> model, feel free to try to come up with a better one.

I disagree that it makes it harder to use, *unless* somebody wants to 
require support for this alternate format.

On the other hand, having an agreed-upon mapping from/to XML will make 
it easier for many developers. Well, at least those who don't have a 
problem with XML :-)

>> The iCalendar format represents a 1990s style approach to the problem.
>> There is no real separation of syntax from the data model. Software
>> developed in that manner is notoriously difficult to get right for the
>> reasons that Keith describes.
>
> XML has lots of problems of its own. I recently had to review a
> specification that referenced WS-i and WS-security and about a couple of
> thousand other pages of useless crap that went with it. All for the sake
> of being able to transmit about six meaningful name-value pairs from a
> client to a server. It was completely ridiculous.

Yes, WS-* sucks. Big news.

> I'm no fan of the iCalendar format, nor the vCard and vCalendar formats
> that preceded it. But for all of its purported generality (and perhaps
> because of it) XML has turned out to be no better, and in many ways
> worse. It's amazing how hard people will work to make a simple idea
> complex, especially when the simple idea has become a bandwagon, or (in
> this case) a religion.
>
> In principle, it would make sense for things to have a uniform syntax.
> But XML gets this wrong in several different ways. The most obvious is
> that XML data structures don't map well onto data structures supported
> by programming languages. That's probably because SGML wasn't designed
> to do that - it was designed to mark up text. Another problem is that
> XML confuses typing and naming. Another problem (especially when mapping
> other structures to/from XML) is that the distinction between parameters
> and sub-elements is pretty much arbitrary.

That may all be true. But strangely enough, there doesn't seem to be 
much energy to come up with something better *and* get people to agree 
on that.

Turning a type registration request into an to-XML-or-not-to thread 
doesn't appear to be productive to me.

>> XML is a substantial overhead if you are dealing with a single
>> protocol but when you are dealing with multiple protocols the benefits
>> are substantial and allow something like 70% of your coding effort to
>> be pushed onto the platform layer. That means that you have 70% less
>> new code and new code paths to contend with.
>
> If your programmer is spending 70% of his coding time dealing with a
> presentation layer, even one as convoluted as iCalendar, you should fire
> him. It's not like regular expression parsers are hard to come by these
> days. Nor are libraries that can parse standard formats hard to come by.

Can iCalendar be parsed reliably with regexps? (just asking).

> Another of the big problems with the XML religion is that its adherents
> have the mistaken impression that defining the syntax is most of the
> work of defining a protocol - so that once you decide to use XML, most
> of the work is done. Apparently, semantics don't matter much.

Another big problem with the anti-XML religion is that its adherents 
like to over-generalize, such as deducing badness of XML from 
WS-deathstar encounters.

Yes, there's lots of bad use of XML. The same can be said about other 
formats as well.

> ...
> It enforces consistency in syntax. Taken by itself, that's a good thing.
> But when you parse XML you don't get a very usable data structure. You
> get a mess. And once you do the work of transforming that data structure
> into an effective internal representation, you've negated whatever
> advantage you might have found in not having to have written a
> parser/generator for it. You haven't solved the problem of needing a
> parser and generator - you've just moved it. Instead of parsing text
> you're parsing a DOM structure. You've added an extra layer or two for
> no benefit.

The benefit is that the XML parser already has taken care of many things 
people get wrong all the time; character encodings, escaping, line 
endings etc.

> You're essentially arguing the syntax by which data is represented on
> the wire - the presentation layer - should constrain how data is
> represented internally in a system. And then you're arguing that the
> particular constraints imposed by XML are appropriate constraints.
> That's brain damage.

I don't think anybody argued that.

>> Now I would quite prefer to take about 50% or more of the XML spec and
>> discard it. They did a good job of taking out the most insane features
>> of SGML but there is much more cruft that could be cut out. But that
>> does not change the fact that using XML as is produces clearer
>> specifications that are more likely to be implemented without errors
>> than with the 1990s approach.
>>

OK, enough :-)

Can we please switch to a discussion of the RFC publication format now?

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 08:23:59 -0400

On Sep 10, 2010, at 6:11 AM, Julian Reschke wrote:

> That's *far* from clear. Could you elaborate in the context of calendaring?

Sigh.  Do I have to write the code to parse the XML and do something with it to show that it's tortured?

> As far as I can tell, the torture is more about having to parse what we have today.

Granted, iCalender is ugly.  But you still have to parse it if you want to interoperate with the installed base. 

> Yes, WS-* sucks. Big news.

I mention it because it's a good example of the "everything should be XML" disease, and a "2010s style" approach to the problem.  Things were better in the 1990s.

> That may all be true. But strangely enough, there doesn't seem to be much energy to come up with something better *and* get people to agree on that.

Give it time.  Of course, now that there is all of this XML to displace, it's hard for something new to get traction.

> Turning a type registration request into an to-XML-or-not-to thread doesn't appear to be productive to me.

Fortunately, we don't need to.  iCalendar already exists, and is not XML.  There's no demonstrated need for a new type, whether in XML or any other syntax.

> Can iCalendar be parsed reliably with regexps? (just asking).

WIthout actually doing the exercise, yes I think so.  I don't recall much (if any) recursion in iCalendar.  Of course the real question is how easy it is to translate ABNF into code that uses regexps.  

(I do think it's ironic that we went to all of this trouble way back in the DRUMS WG to produce a new version of ABNF that would allow the syntax of something to be completely specified - without resorting to comments - and machine parseable.   And then we went to the trouble to rewrite a number of protocols using the new ABNF, accepting the pain and the possibility to introduce errors.   And now that we have it, we're not using it to generate parsers, and people are claiming that they need to use XML instead.  I suppose the one thing that XML adds here is that you don't have to design an internal data structure.  You can use the one that generates naturally from the XML even though it's likely to be very awkward to use.)

> Another big problem with the anti-XML religion is that its adherents like to over-generalize, such as deducing badness of XML from WS-deathstar encounters.

Fair enough - but I do think it's a good example of what happens when people insist that everything be XML.

> Yes, there's lots of bad use of XML. The same can be said about other formats as well.

And I'm not arguing that iCalendar is a well-designed syntax.   If we were building it today, I'd be okay with using XML (not because I like XML, but because it would be politically expedient, and the chances are good than an ad hoc format designed by a WG today would be no better).   My objection is to having two calendar formats.

> The benefit is that the XML parser already has taken care of many things people get wrong all the time; character encodings, escaping, line endings etc.

Yes, but you can get some of those things wrong (e.g. character encodings, and other differences between internal/external representation) internally to your program.

> I don't think anybody argued that.

That's a consequence of the argument that 95% of formats should be XML.  Which was the justification for having an XML version of iCalendar.

Here's the argument in a nutshell: XML or no XML, a new iCalendar format should require substantial technical justification because of the harm it will do to interoperability.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 09:30:32 -0400

> http://fanf.livejournal.com/104586.html

See also http://network-heretics.com/blog/?p=42

Keith

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 09:40:43 -0400

Hi Tony,

> http://fanf.livejournal.com/104586.html

That is a beef about timezones - one piece of iCalendar, but my no means 
all of it.

I disagree with your suggestion of using a location to represent a timezone 
- there are a number of reasons why that won't work - not least that many 
events often do not have a physical location associated with them.

That said work is going on to move to a "timezone by reference" rather than 
"timezone by value" model for iCalendar. There are many driving forces 
behind that, including several of the points you mention, plus the fact 
that often times the timezone definition included in iCalendar data is 
larger than (character-wise) then the actual event information, wasting 
bandwidth.

To that end several options are being proposed. We have already posted a 
draft for a generic timezone service 
(<https://datatracker.ietf.org/doc/draft-douglass-timezone-service/>) - a 
server that can be queried for timezone definitions based on standard IDs 
(Olson identifiers). This allows clients and servers to get timely updates 
to timezone information (rather than having to wait for the next OS 
upgrade). We are working on defining how "timezone by reference" will work 
with the CalDAV calendar access protocol (RFC4791). Since that uses HTTP, 
simple client/server negotiation options exist to facilitate that.

Note that the timezone service is not intended to just be used by iCalendar 
related tools - but we expect/hope that any device that needs to cache 
timezone definitions (e.g. unix zoneinfo data) can also make use of that.

Best place to continue this particular aspect of the discussion would be 
the ietf-calsify WG mailing list.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 09:50:47 -0400

Hi Julian,

> Can iCalendar be parsed reliably with regexps? (just asking).

The Python vobject library that we use in our CalDAV server 
(http://calendarserver.org) does use regular expressions for parsing entire 
lines (after applying line "unfolding" of the overall text object), and 
then parsing out pieces of those. That is used for both iCalendar and vCard.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 09:56:53 -0400

Hi Keith,

--On September 10, 2010 1:40:16 AM -0400 Keith Moore <[email-masked]> 
wrote:

> But here's the acid test.  If you can define a mapping from iCalendar to
> XML that doesn't require any string constants to describe it (other than
> for iCalendar keywords that imply nesting, and separators that are used
> in a regular fashion in iCalendar), and if you can define the inverse
> mapping from XML to iCalendar without naming more than a couple of
> specific element or parameter names - then I'll concede that the mapping
> will probably continue to work in the face of extensions to the iCalendar
> data model.   Otherwise, I'm highly dubious.

That is precisely the goal of draft-daboo-et-al-icalendar-in-xml. iCalendar 
components, properties, parameters and values all map to XML in a 
consistent manner with no need to "special case" based on type or value. 
New components, properties, parameters, values, either registered with IANA 
or using X- prefixes map in exactly the same way.

Conversion to/from XML is trivial - I have coded at least one half of that 
and I know others who have done both ways.It should also be easy to put 
together an XSLT to go from XML to iCalendar - with the only possible 
difficulty being having to apply escaping and line folding as required by 
iCalendar.

-- 
Cyrus Daboo

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 10:09:11 -0400

On Sep 10, 2010, at 9:56 AM, Cyrus Daboo wrote:

> Hi Keith,
> 
> --On September 10, 2010 1:40:16 AM -0400 Keith Moore <[email-masked]> wrote:
> 
> That is precisely the goal of draft-daboo-et-al-icalendar-in-xml. iCalendar components, properties, parameters and values all map to XML in a consistent manner with no need to "special case" based on type or value. 

But you're not doing that in the draft.  You explicitly list every keyword.  Every time a new component, property, parameter, etc is added to iCalendar, the mapping code will have to change also.  The trick is to be able to translate between formats, with no changes to the code needed even when the format is extended.

> Conversion to/from XML is trivial - I have coded at least one half of that and I know others who have done both ways.It should also be easy to put together an XSLT to go from XML to iCalendar - with the only possible difficulty being having to apply escaping and line folding as required by iCalendar.

That would be great, again provided you don't have to explicitly list every keyword.  It still wouldn't convince me that XML iCalendar is sufficiently valuable to be worth the degraded interoperability.  But if you're going to have two formats, being able to automatically convert between them is the best way to do that.

Keith

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 10:24:10 -0400

Hi Keith,

--On September 10, 2010 10:09:11 AM -0400 Keith Moore <[email-masked]> 
wrote:

>> That is precisely the goal of draft-daboo-et-al-icalendar-in-xml.
>> iCalendar components, properties, parameters and values all map to XML
>> in a consistent manner with no need to "special case" based on type or
>> value.
>
> But you're not doing that in the draft.  You explicitly list every
> keyword.  Every time a new component, property, parameter, etc is added
> to iCalendar, the mapping code will have to change also.  The trick is to
> be able to translate between formats, with no changes to the code needed
> even when the format is extended.

Fair enough. We can adjust e.g. Section 3.7 that talks about only X- 
extensions to also refer to any new iCalendar data objects. The basic 
premise being that new iCalendar data object names map directly to an XML 
element name. After each table in the previous sections we can add a 
reference to section 3.7 with a statement that that is how new items will 
be handled.

>> Conversion to/from XML is trivial - I have coded at least one half of
>> that and I know others who have done both ways.It should also be easy to
>> put together an XSLT to go from XML to iCalendar - with the only
>> possible difficulty being having to apply escaping and line folding as
>> required by iCalendar.
>
> That would be great, again provided you don't have to explicitly list
> every keyword.  It still wouldn't convince me that XML iCalendar is
> sufficiently valuable to be worth the degraded interoperability.  But if
> you're going to have two formats, being able to automatically convert
> between them is the best way to do that.

It has always been our intent to have a 1-to-1 mapping between the two. We 
in fact went out of our way to ensure that the XML schema would not 
restrict in anyway what could be done with new iCalendar data objects.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 10:45:46 -0400

Hi Keith,

--On September 9, 2010 10:51:10 PM -0400 Keith Moore <[email-masked]> 
wrote:

> Somebody needs to talk some sense into those people.  Defining another
> calendar format will only harm interoperability.  It doesn't save any
> calendar implementation from needing to implement another parser, because
> if it wants to interoperate with existing products or be able to read old
> events it's still going to have to support iCalendar and probably
> vCalendar also.  So what's the _technical_ (not political) benefit from
> doing this?

First of all look at the mess we have got into with contacts (see recent 
discussion on the vcarddav WG mailing list). There we now have vCard, PoCo, 
OpenSocial and some new thing the OMA is doing (and their are lots of 
private apis too). Yet vCard has been around for a long time - why didn't 
those other folks just use that or at least propose fixes or extension to 
vcard that would satisfy their issues? Well, certainly in the case of PoCo 
one clear requirement was for a simple web/browser based solution - so they 
designed JSON and XML representations.

The same mess could happen to iCalendar too. In fact there are already lots 
of examples of "private" apis using XML to represent calendar data, see 
e.g. Google GDATA. Plus I know of some public event services that uses 
their own custom XML format for event publishers to submit calendar data to 
them. In fact the initial impetus for this work did come from one such 
public events company that really wanted to use iCalendar rather than some 
home grown solution, but really wanted that as XML. Sure you can argue that 
they should be made to parse iCalendar format, but why should they have to 
write their own new parser when they already have reliable, efficient XML 
processing available.

Frankly the important thing here is the semantics of the iCalendar model, 
not the syntax. I want developers to only have to concentrate on getting 
the semantics right without worry too much about the syntax.

Now I can understand your concern about poor interoperability when it comes 
to having the two data formats. I think for the HTTP-based world (CalDAV, 
iSchedule, web-services) this is really not a big deal given the content 
negotiation options. For something like iMIP (iCalendar in email) it is a 
much bigger problem. Speaking for myself only, and not my co-authors, I 
think I would be OK with adding a statement that iMIP messages SHOULD NOT 
use the application/calendar+xml format, or at the very least require both 
text/calendar and application/calendar+xml in e.g. a multipart/alternative 
(though I would worry that the later would cause existing clients problems, 
and lead to message bloat).

> IETF's job is to provide technical leadership, not to follow bad advice.
> Instead of caving into them, what we need to do is publish a draft called
> "Translating Everything into XML Considered Harmful".

Ah, yes. I seem to recall something similar for the use of HTTP - perhaps 
the author of RFC 3205 would like to author this new document too :-)

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Fri, 10 Sep 2010 16:59:34 +0200

On 10.09.2010 16:45, Cyrus Daboo wrote:
> Ah, yes. I seem to recall something similar for the use of HTTP -
> perhaps the author of RFC 3205 would like to author this new document
> too :-)
> ...

In which case I'd be tempted to follow up with

"Inventing unnecessary new protocols, URI schemes and media types 
considered harmful as well - find a good balance, please"

:-)

---

From: [email-masked]
Date: Fri, 10 Sep 2010 07:48:26 -0700 (PDT)

> RFC 5784 might well be a valiant effort toward that.  But it has only been
> out a few months, so I think the jury is still out as to how well it works in
> practice.

I'm still not hearing specifics as to how such a failure can occur, either
in the Sieve XML mapping or the iCal one.

Like most engineering problems, the devil is in the details, which aren't
addressed in any useful fashion by sweeping pronouncements about what
should or should not be attempted.

A mapping to and from XML can be done well or done badly. A good mapping
doesn't require schema and mapping tool changes to accomodate extensions, a bad
mapping does.

Let's look at a specific example. An 'fileinto "foo"' action in Sieve is mapped
by RFC 5784 to XML of the form:

    <action name="fileinto"><str>foo</str></action>

Suppose we had chosen;

    <fileinto>foo</fileinto>

instead. Or maybe:

   <action name="fileinto" arg="foo"/>

Both of these mappings are much simpler and easier to read, but they have a
major failing: The former requires a schema change in order to add new actions,
and the latter requires a schema change to accomodate any sort of extension to
fileinto. By using XML elements to represent the syntactic structure of Sieve
only, we create a situation where only a change to the fundamental syntax of
Sieve - something the base specification specifically prohibits extensions from
doing - is capable of forcing a change to the schema.

iCal has a similar core syntax to Sieve that extensions have to fit into and
cannot change. It follows that as long as that's all you try and represent
using XML elements, extensions will not necessitate a schema change and it
should be possible to write tools to perform the mapping that don't need to  be
updated when extensions are defined.

Unfortunately, now that I've had a chance to look at
draft-daboo-et-al-icalendar-in-xml-06.txt a little more, I find that it doesn't
do this well. For example, instead of mapping a property like dtstamp to
something like:

    <property name="dtstart">
      <date-time>20080205T191224Z</date-time>
    </property>

it maps it to:

       <dtstamp><date-time>20080205T191224Z</date-time></dtstamp>

This means that additional properties will necessitate a schema update. Not
good, and I believe this needs to be fixed.

> I also think it helps that RFC 5784 specifically says it's not intended as an
> alternate storage format for Sieve.  I can certainly see some utility in being
> able to use XML tools to manipulate things not originally written in XML.

It helps in the sense that it makes it clear who needs to support the alternate
format and who doesn't. But it has no effect at all on the issue of  how the
mapping accomodates extensions.

> But here's the acid test.  If you can define a mapping from iCalendar to XML
> that doesn't require any string constants to describe it (other than for
> iCalendar keywords that imply nesting, and separators that are used in a
> regular fashion in iCalendar), and if you can define the inverse mapping from
> XML to iCalendar without naming more than a couple of specific element or
> parameter names - then I'll concede that the mapping will probably continue to
> work in the face of extensions to the iCalendar data model.   Otherwise, I'm
> highly dubious.

I agree that this is what's needed. I don't think it's that difficult to do
though. And I don't think it is difficult to evaluate whether or not that goal
has been reached: All you have to do is look at the extensibility model, and
make sure that anything that conforms to the extensibility model isn't going to
require a schema change.

> from the need to have multiple formats.

It's just not this simple. The loss in interoperability may be made up for by
the increase in accessibility by a different and very powerful set of tools.

> I disagree, because there are interoperability problems resulting from the
> introduction of an alternate format even if the mappings remain invertable in
> the presence of extensions.

I'm sorry, but proof by assertion still isn't working for me here.

> I think you've just illustrated my point.

Tsk tsk. You're quoting me out of context, having removed the part where I said
that this update could have been eliminated had we chosen to do so. In XML,
there's a tradeoff between what I'll call readability (it's actually quite a
bit more than that) and easy extensibility.

> Though really I'm not as worried about Sieve as I doubt that there's nearly
> as much deployment of Sieve as there is of iCalendar.

There are demonstrably many millions of users using Sieve. The number of iCal
users is  probably several orders of magnitude larger, but regardless, these
are both widely deployed formats.

(Of course most of the people using either Sieve or iCal are unaware that's
what they are using. Which is as it should be.)

> I think it's far more likely that all Sieve implementations can be upgraded to
> support XML, if there's a desire to do that, than that all iCalendar
> implementations can be upgraded to support XML.

I doubt very much that all Sieve implementations will be upgraded to support
both formats, nor was that ever the intent when the XML variant was defined.

> I also think that Sieve<>XML is inherently saner than iCalendar<>XML because
> Sieve (being essentially a programming language) is already structured
> similarly to the way XML is structured.

Sorry, I'm not seeing the major distinction here. It's just neesting of
structures in either case.

> I think that the burden is on the proposers to justify producing an
> incompatible alternative to a existing standard, which will definitely harm
> interoperability.

And now you're asking the authors to prove a negative.

> If people want to define mappings between iCalendar and XML just for the sake
> of being able to manipulate the data using XML tools, that doesn't bother me so
> much.   It's when people want to start exchanging calendar data using XML in
> some cases and iCalendar in others that the interoperability problems start
> cropping up.  (Though I have to wonder how much it really helps to be able to
> manipulate calendar data using XML tools, as calendar manipulations tend to be
> fairly specialized to calendar applications.)

Even if XML-specific tools like stylesheets prove less than useful in
performing manipulations of calendar data, there's still significant benefit
associated with being able to use built in parsing capbilities, espcially when
those capabilites are nicely tied to automatic creation of complex data
stuctures in various languages.

				Ned

---

From: Julian Reschke <[email-masked]>
Date: Fri, 10 Sep 2010 18:04:43 +0200

> good, and I believe this needs to be fixed.

Not really. How is adding a new allowed value to an XML attribute any 
different from adding a new element name (assuming a schema language 
than can express both constraints?).

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 12:11:40 -0400

Hi Ned,

--On September 10, 2010 7:48:26 AM -0700 Ned Freed <[email-masked]> 
wrote:

> draft-daboo-et-al-icalendar-in-xml-06.txt a little more, I find that it
> doesn't do this well. For example, instead of mapping a property like
> dtstamp to something like:
>
> This means that additional properties will necessitate a schema update.
> Not good, and I believe this needs to be fixed.

There was a lot of debate over the choice of element names vs element 
attributes to name the iCalendar data objects. Really the key to your 
argument is whether or not it is vital to be able to validate the XML via 
the schema - I think our contention was that was not a requirement. More 
important is validating the semantics of the calendar data (e.g. DTEND 
always >= DTSTART, ORGANIZER always present in iTIP messages) and I believe 
no one is going to want to do that via the XML schema.

If you feel strongly about this I suggest you bring this issue up on the 
vCardDAV WG mailing list wrt to the vcard-in-xml spec which uses the same 
design as icalendar-in-xml. If there is agreement there to change, then we 
would likely re-align icalendar-in-xml to match.

-- 
Cyrus Daboo

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 12:43:53 -0400

On Sep 10, 2010, at 10:24 AM, Cyrus Daboo wrote:

> Hi Keith,
> 
> --On September 10, 2010 10:09:11 AM -0400 Keith Moore <[email-masked]> wrote:
> 
> Fair enough. We can adjust e.g. Section 3.7 that talks about only X- extensions to also refer to any new iCalendar data objects. The basic premise being that new iCalendar data object names map directly to an XML element name. After each table in the previous sections we can add a reference to section 3.7 with a statement that that is how new items will be handled.

That would help.  But why do you need specific rules for any of the iCalendar data object names?  I can understand one or two exceptional cases, but if the mapping you have is truly adaptable, it shouldn't need many of those rules.

Keith

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 10 Sep 2010 12:49:57 -0400

Hi Keith,

--On September 10, 2010 12:43:53 PM -0400 Keith Moore 
<[email-masked]> wrote:

>> premise being that new iCalendar data object names map directly to an
>> XML element name. After each table in the previous sections we can add a
>> reference to section 3.7 with a statement that that is how new items
>> will be handled.
>
> That would help.  But why do you need specific rules for any of the
> iCalendar data object names?  I can understand one or two exceptional
> cases, but if the mapping you have is truly adaptable, it shouldn't need
> many of those rules.

OK, I'll discuss with my fellow authors to see what we can do to better 
clarify our intent. Maybe we actually state the naming/mapping rules at the 
very start of section 3 as the "fundamental" set of rules, but we still 
keep the tables in there as a useful reference back to the iCalendar spec.

-- 
Cyrus Daboo

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 13:21:25 -0400

> First of all look at the mess we have got into with contacts (see recent discussion on the vcarddav WG mailing list). There we now have vCard, PoCo, OpenSocial and some new thing the OMA is doing (and their are lots of private apis too). Yet vCard has been around for a long time - why didn't those other folks just use that or at least propose fixes or extension to vcard that would satisfy their issues? Well, certainly in the case of PoCo one clear requirement was for a simple web/browser based solution - so they designed JSON and XML representations.

Mumble.  It's a lot harder to make a web browser do something useful with a calendar object (no matter what the syntax) than it is to make a web browser do something useful with contact information.  

And I *like* JSON.  I think it's a good approximation to what XML should have been. 

And I can even see that having a DOM-based representation of calendar data for internal use in a program, as awkward as that is, might be better than lots of ad hoc internal representations.   (I can't quite put my finger on it, but there does seem to be something about iCalendar that encourages building poor internal representations of the data.)

Still, if I were designing a PIM that wanted to be able to have web browsers manipulate contact information, I'd use vCard as the interchange format and use JSON only for the purpose of communicating with Javascript code in the browser.

> The same mess could happen to iCalendar too. In fact there are already lots of examples of "private" apis using XML to represent calendar data, see e.g. Google GDATA. Plus I know of some public event services that uses their own custom XML format for event publishers to submit calendar data to them. In fact the initial impetus for this work did come from one such public events company that really wanted to use iCalendar rather than some home grown solution, but really wanted that as XML. Sure you can argue that they should be made to parse iCalendar format, but why should they have to write their own new parser when they already have reliable, efficient XML processing available.

Code reuse is all the rage these days.  Can't they leverage work that someone else has already done?  

> Frankly the important thing here is the semantics of the iCalendar model, not the syntax. I want developers to only have to concentrate on getting the semantics right without worry too much about the syntax.

As much as I'd like to believe you can separate the model from the syntax, experience seems to indicate otherwise.  Every time I've seen people borrow a data model from one protocol and move it to another, the data models used by the two protocols have diverged.    Think about the differences between interpretation of email header fields and HTTP header fields.  The field names get borrowed, the syntax of each field generally stays the same, but the semantics change. 

And there are reasons why the semantics change.  In the case of calendar data, consider timezone handling.  If you're sending out an event "in the blind" (say over email) without knowing whether the recipient will recognize your event timezone, you want to send as much mapping data as the receiver will need - even though it might change.  But if you're sending the event over some link that permits interaction and negotiation, you probably want to either ask the receiver whether it supports the timezone, or give it a link to the timezone data so that it can refresh it as needed.

This also reminds me of why IETF avoids standardizing APIs: it's because people tried to use the API (which is pretty much just a data model) as the basis for interoperability rather than bits on the wire.  And it didn't work.

> Now I can understand your concern about poor interoperability when it comes to having the two data formats. I think for the HTTP-based world (CalDAV, iSchedule, web-services) this is really not a big deal given the content negotiation options. For something like iMIP (iCalendar in email) it is a much bigger problem. Speaking for myself only, and not my co-authors, I think I would be OK with adding a statement that iMIP messages SHOULD NOT use the application/calendar+xml format, or at the very least require both text/calendar and application/calendar+xml in e.g. a multipart/alternative (though I would worry that the later would cause existing clients problems, and lead to message bloat).

Something like that would help, I think.  It seems like the appropriate goal is to give clear guidance to implementors about when to use iCalendar and when to use the other formats.  I think there's a case to be made for standardizing on a single format for (a) sending "in the blind" when you can't do content-negotiation, and (b) as a "must support" format (even if you can negotiate others, you must support the base format, so that every implementation has a basis for interoperation).    And I suspect that iCalendar has the advantage as the "must support" format; though somebody might like to argue the opposite case.

> Ah, yes. I seem to recall something similar for the use of HTTP - perhaps the author of RFC 3205 would like to author this new document too :-)

perhaps.

Keith

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 10 Sep 2010 11:38:21 -0600

On 9/10/10 11:21 AM, Keith Moore wrote:
> 
>>>> An XML representation for iCalendar is vital if we are to keep
>>>> iCalendar relevant in the web-based world. The drive for this
>>>> work comes from a number of areas - in particular the smart
>>>> grid effort sponsored by NIST will make use of this as part of
>>>> the standards suite they are defining.
>>> 
>>> Somebody needs to talk some sense into those people.  Defining
>>> another calendar format will only harm interoperability.  It
>>> doesn't save any calendar implementation from needing to
>>> implement another parser, because if it wants to interoperate
>>> with existing products or be able to read old events it's still
>>> going to have to support iCalendar and probably vCalendar also.
>>> So what's the _technical_ (not political) benefit from doing
>>> this?
>> 
>> First of all look at the mess we have got into with contacts (see
>> recent discussion on the vcarddav WG mailing list). There we now
>> have vCard, PoCo, OpenSocial and some new thing the OMA is doing
>> (and their are lots of private apis too). Yet vCard has been around
>> for a long time - why didn't those other folks just use that or at
>> least propose fixes or extension to vcard that would satisfy their
>> issues? Well, certainly in the case of PoCo one clear requirement
>> was for a simple web/browser based solution - so they designed JSON
>> and XML representations.
> 
> Mumble.  It's a lot harder to make a web browser do something useful
> with a calendar object (no matter what the syntax) than it is to make
> a web browser do something useful with contact information.
> 

It's not clear to me how your personal preference for JSON over XML is
relevant to registration of the application/calendar+xml media type or
continued work on draft-daboo-et-al-icalendar-in-xml. However, given
that you seem to object to the XML representation of vCards, calendaring
objects, and just about anything else, I suggest that you start a thread
about XML vs. JSON on the apps-discuss list rather than filling up the
archives of the ietf-types and [email-masked] lists.

https://www.ietf.org/mailman/listinfo/apps-discuss

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: [email-masked]
Date: Fri, 10 Sep 2010 10:12:15 -0700 (PDT)

> than can express both constraints?).

First of all, there are many situations where you don't get to pick your schema
lannguage. (In fact I'd say it's the rule rather than the exception.) So even
there are Schema languages that support "any element can appear here" - the
fact that XML Schema doesn't allow this (xs:any has far too many restrictions
placed on it to be usefullly usable) means you shouldn't be defining things
this way.

Second, it is of course possible to impose restrictions on the values that can
appear in an attribute value (or element content for that matter). But just
because you can doesn't mean you should. And these sorts of mappings are
exactly the place where you shouldn't be doing this, at all, ever, because when
you do you end up having to change the schema for each extension that adds
properties (in the case of iCal) or actions/tests/controls (in the case of
Sieve).

Furthermore, in the case of Sieve at least, there are cases where it is
entirely valid to specify tests and actions in a script that the implementation
you're currently using doesn't support. For example:

    require "ihave";
    if ihave "ereject" { ereject "I hate this message"; }
    elsif ihave "reject" { reject "I still hate this message";}
    else { discard; }

or:

    require "ihave";
    if ihave "x-private-extension-nobody-else-has-heard-of" {
      if bletch "bar" "foo" { frob :zing "foo" 1 "bar";}
    }

If I've mapped actions and tests to elements here I'm screwed - there's no way
to make this work properly in XML Schema without either adding a ton of
superfluous bracketing elements all over the place (and we already have too
many of these), or creating a false distinction between extensions and base
elements that is, if anything, even uglier.

It's also easy to show that full script validity checking here requires solving
the satisfiability problem, which means its in NP. I'm pretty sure that exceeds
the capabilities of most schema languages ;-)

I believe similar issues exist in iCal and probably vCard.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Fri, 10 Sep 2010 13:43:34 -0400

On Sep 10, 2010, at 12:39 PM, Phillip Hallam-Baker wrote:

> 
> 
> On Fri, Sep 10, 2010 at 10:48 AM, <[email-masked]> wrote:
> 
> What many XML-haters do not understand is that the syntax is designed to completely automate the process of writing the parser and the backing data classes.

I, at least, understand that.  But in my experience the backing data classes generally end up being ridiculously baroque.

> Starting with an XML Schema definition I can generate the corresponding data structures automatically with one mouse click together with the corresponding parser/serializer calls.
> 
> 
> Starting from an EBNF description, I have to first read the description. This has already taken more time than working with the XML version would.

Really you should be able to generate the parser from the *BNF.   But you're of course correct that it won't generate the data structures for you.   All you get is a state machine that recognizes the language.

> I then have to work out if the grammar is an FSM or LR(1) or something else. When I was a grad student I used to write yacc parsers but these days I have written enough parser generators that I can actually hand code quicker than it takes me working round the peculiarities of yacc.

Biggest problem I had with using yacc based parsers was doing good error recovery, because people would inevitably generate data representations that didn't quite fit the grammar.  But XML parsers tend to not be much better at that than yacc is.   With both XML and yacc, either the entire message is recognized or it isn't, unless you do a fair bit of work (often redefining the language in the process).

> So what takes me no time at all with XML is likely to take a couple of days and considerably more skill with EBNF.

Granted.  But surprisingly, this actually comes at a cost.  When every protocol had its own syntax for representing data, there was pressure to keep the data models from getting too complex.  With XML, people are free to make the data models as complex as they want - and often, this means that the data models are not well thought out.   That's not to say that having a standard, uniform presentation syntax which translates readily to and from an internal data structure is a bad thing.   (Even if I don't like XML specifically, I think the underlying goal is sound.)  What it means is that protocol designers and data model designers still need to exercise discipline to keep their data models simple.

(not that any of the above has much to do with iCalendar, since the data model there is already largely determined.)

Keith

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 10 Sep 2010 11:36:34 -0700

On 9/9/2010 8:38 PM, Ned Freed wrote:
> This is demonstrably false.

We need to distinguish between alternate syntactic forms, versus alternate 
semantic environments.  Translating between versions of the former do not need 
to lose information.  Translating between versions of the latter almost 
certainly do.  Losing information is about differences in semantics.

As I understand the calendar+xml, it is "merely" a syntactic alternative.  To 
the extent that it requires information loss when being re-encoded, yes that 
should be fixed.  But it's not likely to be difficult and the existence of two 
syntactic forms is not inherently problematic.  (We have lots of examples on the 
net of doing this quite nicely, at different layers of Internet architecture.)

As for the more abstract discussion about whether it's good or bad to have an 
xml version, I'll strongly suggest that it is best conducted in a real bar bof 
with real alcohol.  (I'll be supporting its existence, FWIW.)  The xml version 
is an important fact of life.  Let's not pretend otherwise.

It is not the job of the MIME registration process to make political statements 
that give preferential treatment to facts of life that some might like more than 
other facts of life...

Register the damn thing.  The registration form appears to satisfy registration 
requirements.

If there are specific problems with the associated spec, pursue them 
independently and concretely, please.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 10 Sep 2010 00:34:46 -0400

The objections raised by Keith do not appear to me to fall under any of the
requirements for MIME type registration set out in RFC 4288.

If you want to have a system in which 95% of your data structures are in XML
you probably don't want to have to introduce a separate syntax and you most
certainly do not want to deal with a separate data model for dealing with
calendaring.

The iCalendar format represents a 1990s style approach to the problem. There
is no real separation of syntax from the data model. Software developed in
that manner is notoriously difficult to get right for the reasons that Keith
describes.

XML is a substantial overhead if you are dealing with a single protocol but
when you are dealing with multiple protocols the benefits are substantial
and allow something like 70% of your coding effort to be pushed onto the
platform layer. That means that you have 70% less new code and new code
paths to contend with.

One of the discoveries of the mid 1990s was that yacc and LR(1) grammars are
no more useful for describing computer languages than they are for
describing natural languages. The most useful feature of a computer grammar
is regularity and consistency. XML enforces a high degree of consistency.

Now I would quite prefer to take about 50% or more of the XML spec and
discard it. They did a good job of taking out the most insane features of
SGML but there is much more cruft that could be cut out. But that does not
change the fact that using XML as is produces clearer specifications that
are more likely to be implemented without errors than with the 1990s
approach.

Website: http://hallambaker.com/

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 10 Sep 2010 07:59:18 -0400

> Can we please switch to a discussion of the RFC publication format now?

If people do not like my approach to designing protocols and want to
persuade the market to adopt a different approach they should consider
adopting my proposals for making the RFC publication format more likely to
be influential.

The reason we use XML is to achieve Bill Gate's vision of 'frictionless
capitalism'. Tests conducted by the World Wide Web Consortium in the MIT
wind tunnel conclusively demonstrated that the angle brackets have the
lowest drag coefficient of any ASCII punctuation character.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 10 Sep 2010 11:12:09 -0400

At one time there were people who whined one endlessly about the evils of
HTML email.

And before that there was a guy who used to drone on endlessly in comp.arch
about how high level languages were a mistake and how everything should be
written in machine code.

The technology world moves on. Either deal with it or open a museum.

On Fri, Sep 10, 2010 at 10:59 AM, Julian Reschke <[email-masked]>wrote:

> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 10 Sep 2010 12:39:57 -0400

On Fri, Sep 10, 2010 at 10:48 AM,
<[email-masked]<[email-masked]>

> stuctures in various languages.

What many XML-haters do not understand is that the syntax is designed to
completely automate the process of writing the parser and the backing data
classes.

Starting with an XML Schema definition I can generate the corresponding data
structures automatically with one mouse click together with the
corresponding parser/serializer calls.

Starting from an EBNF description, I have to first read the description.
This has already taken more time than working with the XML version would.

I then have to work out if the grammar is an FSM or LR(1) or something else.
When I was a grad student I used to write yacc parsers but these days I have
written enough parser generators that I can actually hand code quicker than
it takes me working round the peculiarities of yacc.

So what takes me no time at all with XML is likely to take a couple of days
and considerably more skill with EBNF.

Of course this approach works best in modern languages like Java and C# but
I have generated similar tools for C and I am pretty sure the same tools
exist for objective C. Its going to suck somewhat if you are coding in
FORTRAN or Pascal.

---

From: Keith Moore <[email-masked]>
Date: Sat, 11 Sep 2010 00:56:03 -0400

[ietf-types removed to spare those who just want to read type applications.]

> We need to distinguish between alternate syntactic forms, versus alternate semantic environments.  Translating between versions of the former do not need to lose information.  Translating between versions of the latter almost certainly do.  Losing information is about differences in semantics.

I'm not convinced that there is a difference in practice between the two.  Different syntactic forms tend to get used in different environments, and to adapt to the requirements/conventions of those environments.  

calendar+xml is intended to be merely a syntactic alternative.  I just don't believe it is likely to stay that way.

> As I understand the calendar+xml, it is "merely" a syntactic alternative.  To the extent that it requires information loss when being re-encoded, yes that should be fixed.  But it's not likely to be difficult and the existence of two syntactic forms is not inherently problematic.  (We have lots of examples on the net of doing this quite nicely, at different layers of Internet architecture.)

please cite specific examples.  it might be instructive to see why they work or don't work well.

the best one that immediately comes to mind is raw IP vs. PPP with header compression.  I think it works because the latter representation is only used on the wire between two endpoints of a network link.  And if it fails to faithfully reproduce the packet, it's very clear where the problem is.  (there's no argument about which representation is correct - the original packet is always correct.).

> As for the more abstract discussion about whether it's good or bad to have an xml version, I'll strongly suggest that it is best conducted in a real bar bof with real alcohol.  (I'll be supporting its existence, FWIW.)  

I had to read that twice.  For a second I thought you were offering to support the existence of alcohol for the bar bof. 

> The xml version is an important fact of life.  Let's not pretend otherwise.

My standard response whenever anyone cites anything as "reality" or a "fact of life", is that that's what people always say when they can't cite any technical justification.  

Keith

---

From: Nathaniel Borenstein <[email-masked]>
Date: Sat, 11 Sep 2010 08:42:13 -0400

Although I agree, and will stipulate, that:

	-- Multiple syntaxes are nearly always a bad idea

and

	-- iCal is badly broken (in fact, more broken than any of the cited articles indicate)

We don't live in the best of all possible worlds.  There is a demonstrable need to continue to support iCal, and to have an interoperable XML format, until something better comes along.  While I would enthusiastically support and work on such a better calendar format (I will resist the urge to add my list of additional problems with iCal to the discussion), I think we have to cope with current reality, in which there is a growing demand for XML calendar interoperation.

In other words:  iCal sucks but that's irrelevant.  Multiple syntaxes suck, but sometimes they're necessary.  We've created a great big turd, and now we need to periodically squirt perfume on it or it will only smell worse and worse over time.  -- Nathaniel

---

From: Keith Moore <[email-masked]>
Date: Sat, 11 Sep 2010 10:26:37 -0400

On Sep 11, 2010, at 10:20 AM, Phillip Hallam-Baker wrote:

> I don't see any point in having a design conversation based on what might happen to future iterations of the design.

I don't see the point in having a design conversation that isn't based on what might happen to the design.  If the design is carved in stone, the conversation is no longer about design; it's about history.

> It is of course possible, likely even that two variants of a syntax would diverge over time. At some point there will be a decision that future versions of the format will use only one of the syntaxes.

Why wait, and further confuse the market?  We can make that decision now.

> Given the way that calendar applications behave, an incompatible format change would be no bad thing. In particular, calendar applications seem to have no idea that there is a difference between local time and UTC or that daylight savings rules vary with place. I don't know if there is also a problem in the iCal format, but the applications are certainly broken.

Maybe you're using the wrong calendar applications.  The ones I've used haven't had trouble with that.  

(and BTW, the format is iCalendar.  iCal is a Macintosh application)

Keith

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 11 Sep 2010 10:20:02 -0400

I don't see any point in having a design conversation based on what might
happen to future iterations of the design.

It is of course possible, likely even that two variants of a syntax would
diverge over time. At some point there will be a decision that future
versions of the format will use only one of the syntaxes.

The happened in the XML world, at the start most specifications included
both the DTD and XML Schema definitions of the schema. Now we only use the
XML Schema version and DTDs are obsolete.

Given the way that calendar applications behave, an incompatible format
change would be no bad thing. In particular, calendar applications seem to
have no idea that there is a difference between local time and UTC or that
daylight savings rules vary with place. I don't know if there is also a
problem in the iCal format, but the applications are certainly broken.

I suspect that the problem here is the type of fourth rate 'usability'
specialists who have ruined a lot of security protocols by removing
information that the user needs from the user interface. They think that
user acceptance scores in 60 minute lab experiments are definitive. But all
that a short interaction in a lab can show you is the usability of the
application for the task you give them.

Without the ability to specify the local time when setting a recurring
appointment, a calendar application is broken. A meeting that happens at 9am
Eastern time every week is not the same as one that happens at 2pm UK time.

When I was using Outlook it would let me specify the place I was in but
didn't bother to change the time zone for me unless I did it manually and
the setting would then affect the whole machine (which of course is a
privileged operation). So when setting up a trip to the West cost for next
week, Outlook was still trying to find me meeting times in the wrong time
zone.

---

From: [email-masked]
Date: Sun, 12 Sep 2010 20:10:12 -0700 (PDT)

> I'm not convinced that there is a difference in practice between the two. 
> Different syntactic forms tend to get used in different environments, and to
> adapt to the requirements/conventions of those environments.

That has not been my experience. My experience has been that when it comes to
interoperability in both the short and long term, clear, accurate and concise
specifications of the formats and the mappings between them are key.

> believe it is likely to stay that way.

And I believe it will if it is done properly, which in the absence of any
actual evidence is an equally valid statement. Again, the problem with this
entire discussion is that it's mostly happening at the 20,000 foot level,
leaving the details of the protocol - which really matter - behind.

> > As I understand the calendar+xml, it is "merely" a syntactic alternative. 
> > To the extent that it requires information loss when being re-encoded, yes
> > that should be fixed.  But it's not likely to be difficult and the
> > existence of two syntactic forms is not inherently problematic.  (We have
> > lots of examples on the net of doing this quite nicely, at different
> > layers of Internet architecture.)

> please cite specific examples.  it might be instructive to see why they work

Fair enugh, but there are vast numbers of examples of alternate syntactic forms
out there - so many that I'm sure you can find an example to support almost any
assertion you want to make.

So let's narrow things down to formally standardized data formats where
semantic differences were disallowed. I've already discussed Sieve as an
example in this context, but another, older one would be CGM (computer graphics
metafile).

This is actually an instructive case between there were three syntaxes for CGM,
not two: text, binary, and compressed text. (More recently, something called
WebCGM has been specified, but this happened long after my involvement ceased
and I know nothing about it.) Both the text and binary formats were quite
straightforward and clearly specified. Back in the day we had multiple
applications that happily generated either the text or binary format, and at
least one which could generate both. Applications able to process CGM generally
only supported one variant, but there were at least two readily available
software packages that could convert between text and binary. 

But what about the compressed text format? It turned out to be quite
problematic. Part of the problem was obvious: Three formats is pretty clearly
at least one too many. It's enough of a pain to support two formats; three was
seen by implementers as ... excessive. Even worse, the compressed text format
was much more complex, and IMO not terribly well specified. Usage of it was
comparitively rare, and the software that did try and handle it tended to have
gotten minimal testing and was quite buggy. As I recall, we had one application
that only supported compressed text output, and we had a devil of a time
finding something that could process it without crashing. Even worse, things
would work fine until something new was tried, and then more problems would
emerge.

So what are the lessons here? Well, pretty much the same old same old: Format
simplicity and specification clarity are the key to successful
interoperability, far more so than having a single format. In the case of CGM,
converting wasn't the problem, use of the compressed text format in any context
was. Things would almot certainly have been worse had compressed text been the
only format available.

> the best one that immediately comes to mind is raw IP vs. PPP with header
> compression.  I think it works because the latter representation is only used
> on the wire between two endpoints of a network link.  And if it fails to
> faithfully reproduce the packet, it's very clear where the problem is. 
> (there's no argument about which representation is correct - the original
> packet is always correct.).

I don't think the comparison is particularly useful because the domains are so
different, but even so, I think the fact that the mapping is fairly simple and
clearly specified goes a long way to making these protocols interoperate as
well as they do.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Tue, 14 Sep 2010 00:22:26 -0400

Let me say off the bat that I'm now convinced that there is at least some marginal utility in having an XML representation of iCalendar.   And the barrier for registration of new types is deliberately low, so I now think there's enough utility in having such a type.

I still think it would be helpful to improve the mapping so that it's clear that it will work across reasonable changes/extensions to iCalendar format, and to simplify it so that it's easier to understand and implement.  

I also think it would be appropriate for the document to make it clear that the standard for exchange of calendar data between systems is still iCalendar format, and that implementations should still have the ability to import and export iCalendar.

> specifications of the formats and the mappings between them are key.

I certainly agree that if you're going to have multiple file formats, those are the optimal conditions.  If the mapping is simple enough, and the specification clear and concise enough that it's obvious how to write correct code to map between the two, the chance that implementations will support both formats is very good.

And without quoting and responding at length, I think your example of CGM is instructive.

Though I do have to wonder, if the mapping is really that simple, clear, and concise - why is there a need for any implementation to support multiple external formats?  Each system that prefers a tree-structured internal representation can just use that specification to convert at the boundary.

Anyway, I feel like I've made my point and it has been given due consideration, and I hope that the discussion has been useful for the authors and/or IESG.

Keith