Subject: Re: draft-housley-iesg-rfc3932bis and the optional/mandatory nature of IESG notes

From: Jari Arkko <[email-masked]>
Date: Tue, 01 Sep 2009 10:43:22 +0300

Adam,

> So, to be clear, the question you have raised has to do with the 
> difference between:
>
>    "The IESG may choose to add an IESG note to an Independent
>    Submission..."
>
>
> and:
>
>    "The IESG may choose to request the addition of an IESG note to an
>    Independent Submission..."
>
>
>
> Right?

Yes.

> This has nothing to do with the text:
>
>    "In exceptional cases, when the relationship of the document to the
>    IETF standards process might be unclear..." ?

Right. The exceptional vs. always-on question is another issue.

Jari

---

From: <[email-masked]>
Date: Tue, 1 Sep 2009 09:55:00 +0200

Joel M. Halpern wrote:

> If the ISE / RSE is unreasonable, the IAB will slap the editor and say
> "stop doing that."  There is no equivalent process if we reverse the
> structure.

Yes, there is. If the IESG would request/recommend a particularly bad
IESG note, this decision can be appealed just like any other IESG
decision. The IAB would then determine if the IESG acted appropriately
or not.

On the other hand, if the ISE/RSE decides to publish a document
without an IESG note even if the IESG requested/recommended it, this
decision cannot be effectively appealed (since the RFC already came
out, and can't be really "recalled").

Although I'm not expecting this really to happen very often (if ever),
from checks-and-balances viewpoint I would support (b) from Jari's
email (in other words: RFC Editor cannot unilaterally ignore a note
requested by IESG, but has to take it to the IAB via the usual
appeal procedures).
 
Best regards,
Pasi

---

From: John C Klensin <[email-masked]>
Date: Wed, 02 Sep 2009 13:20:20 -0400

--On Tuesday, September 01, 2009 09:55 +0200
[email-masked] wrote:

> Joel M. Halpern wrote:
> 
>> If the ISE / RSE is unreasonable, the IAB will slap the
>> editor and say "stop doing that."  There is no equivalent
>> process if we reverse the structure.
> 
> Yes, there is. If the IESG would request/recommend a
> particularly bad IESG note, this decision can be appealed just
> like any other IESG decision. The IAB would then determine if
> the IESG acted appropriately or not.
> 
> without an IESG note even if the IESG requested/recommended
> it, this decision cannot be effectively appealed (since the
> RFC already came out, and can't be really "recalled").
> 
> Although I'm not expecting this really to happen very often
> (if ever), from checks-and-balances viewpoint I would support
> (b) from Jari's email (in other words: RFC Editor cannot
> unilaterally ignore a note requested by IESG, but has to take
> it to the IAB via the usual appeal procedures).

Pasi,

A comment and then a suggested middle position.

I've been watching what we now call the Independent Submission
Process for far longer than there has been an IETF.   I've seen
it as an insider for a large fraction of two decades -- as an
AD, an IAB member and then chair, as an editorial board member,
and now as an IAB member again.   During that period, I've never
seen an RFC Editor abuse the process by ignoring legitimate
input.  Bob Braden may be able to provide an inside view --not
in his present role of RFC Editor but as the very-long-time IAB
Exec Director -- of what happened before 1992, but my educated
guess is that instances of "RFC Editor ignores input" during
that time were also about zero.

During the same period, I'd seen behavior I consider abusive
from ADs or the IESG many times --  attempting to prevent
publication of documents with which they had personal/ emotional
disagreements that they were unwilling or unable to explain in
public, asking for publication holds on documents for multiples
of years, insisting that the RFC Editor not move forward until
the IESG responds in some way and then not responding for months
and months, demanding changes that would significantly weaken
the document or change its meaning, and so on.  Many of those
problems have been resolved by negotiation, but some have not.
RFC 3932, and its limitation on technical review, was a huge
improvement over its predecessors in that regard, but we've
heard multiple ADs over the years claim that they can redefine
any disagreement about a document into either a technical issue
or a technical one (whichever is needed) if they care enough and
especially if they can define the boundary (which they also have
insisted that the IESG has the unilateral right to do).

In principle, the IAB could appoint a new ISE to take over in
January who would adopt a policy of abusiveness.  But I think I
can speak for the ACEF membership and the IAB if I say that we
don't intend to do that... and that the IAB would expect the RSE
to move fairly quickly, with the IAB's backing, to correct the
attitudes involved if it occurred anyway.

So trying to make IESG notes mandatory on documents originating
in another stream for the reasons you cite above is solving a
problem we've never had at the risk of making a problem worse
that we've had several times.  That strikes me as bad
engineering at best.

And insisting that the RFC Editor invoke a formal appeals
procedure in case of disagreement with the IESG about an
Independent Submission would, as Olaf points out, largely undo
the efforts of the last few years to clearly separate the
different streams.  It would be as sensible to say that IESG
notes should be sufficiently exceptional that the IESG would
need to consult the IAB and get permission before sending any
such note-request to the ISE.  I suspect that such a provision
would not make either the IESG or the IAB very happy.

However, if your concern is really to make sure that there is a
timely appeal path, I have a suggestion that might be acceptable
to everyone without causing unfortunate side-effects.  We simply
require that, if the ISE receives input from the IESG requesting
specific changes to a document ("specific changes" including,
but not limited to, so-called "IESG Notes") and the ISE and
authors decide to not incorporate those proposed changes, the
ISE is required to explain to the IESG, in writing, why not and
allow a reasonable period of time for the IESG to respond.  If
it felt it were necessary, the IESG could then open a further
discussion, ask the RSE to mediate, or launch a formal request
for IAB review.  Consistent with other provisions in RFC 4846,
either the IESG or the ISE could, at their discretion, make the
correspondence (the request and response) public.

The one restriction I'd impose on this is that "reasonable time"
not be more than a few weeks... again, there has been abuse in
that area in the past and re-enabling such abuse would be, well,
dumb.  If the ISE notifies the IESG and the IESG doesn't
respond, then the ISE can go ahead and publish without further
delay.

Does that work for you?  And, if not and you still think you
need mandatory notes, why?

best,
    john

---

From: Sam Hartman <[email-masked]>
Date: Wed, 02 Sep 2009 13:53:49 -0400

>>>>> "John" == John C Klensin <[email-masked]> writes:
    John> However, if your concern is really to make sure that there
    John> is a timely appeal path, I have a suggestion that might be
    John> acceptable to everyone without causing unfortunate
    John> side-effects.  We simply require that, if the ISE receives
    John> input from the IESG requesting specific changes to a
    John> document ("specific changes" including, but not limited to,
    John> so-called "IESG Notes") and the ISE and authors decide to
    John> not incorporate those proposed changes, the ISE is required
    John> to explain to the IESG, in writing, why not and allow a
    John> reasonable period of time for the IESG to respond.  If it
    John> felt it were necessary, the IESG could then open a further
    John> discussion, ask the RSE to mediate, or launch a formal
    John> request for IAB review.  Consistent with other provisions in
    John> RFC 4846, either the IESG or the ISE could, at their
    John> discretion, make the correspondence (the request and
    John> response) public.

John, in principle,  I would be delighted by this option if you made a few more changes to make the RFC process more accountable:

1) Open up the rfc-editorial board so that it was selected by some sort of nomcom/community process.  That nomcom could of course draw from a broader community than the IETF as a whole

2) Provide an appeals path for IAB decisions related to the RFC-editor function

I have a lot more faith in the IETF process than I do the RFC editor
process.  I believe that the RFC editor process is more open to a
different type of abuse than the IESG process, but I believe we have a
far more open process for addressing problems with the IESG than we do
with IAB decisions about the RFC editor or with the RFC editor process
itself.

However, absent these changes, I don't believe there would be
appropriate checks and balances present.

---

From: Bob Braden <[email-masked]>
Date: Wed, 02 Sep 2009 11:50:00 -0700

> process. 

Sam,

If you have a specific complaint about the functioning of the RFC 
Editor, please bring it out on the rfc-interest list.
I don't know what kind of abuse you think we are open to, but I would 
certainly like to hear it.

Bob Braden
>  I believe that the RFC editor process is more open to a
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 02 Sep 2009 15:09:41 -0400

RFC 5620 calls for the appointment of an RFC Series Advisory Group, to 
be appointed by the IAB, and a Independent Submissions Stream Editorial 
Board (ISSEB for now), which serves at the pleasure of the ISE.
This was reviewed and approved by the community.
I presume with cognizance of the existing rules about the authority over 
the Streams.

I presume you are concerned about the membership of the ISSEB.
Given that this stream is specifically not an IETF stream, I do not see 
why it would make sense for the membership to be appointed by the IETF 
community, through its nomcom designates.

With regard to appeals, are you asking for an ability to appeal an IAB 
decision about the RFC Editor?  Presumably, a procedural appeal could be 
made to the ISoC Board of Trustees, if the IAB had not followed 
documented procedures.  But otherwise, we are back to the issue of 
undermining the Independence of the Independent Stream.

Remember, all of these documents are Experimental or Informational.
Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 03 Sep 2009 09:02:50 +1200

Sam,

On 2009-09-03 05:53, Sam Hartman wrote:

...
> 1) Open up the rfc-editorial board so that it was selected by some sort of nomcom/community process.  That nomcom could of course draw from a broader community than the IETF as a whole

I'm certainly in favour of transparency in the process of setting
up the Independent stream's editorial board. I could see value in
an open call for nominations. However, you're asking for it to be
set up in way that's very different from the typical editorial board
for a scholarly journal or the typical technical programme committee
for a scholarly conference. It seems to me that the editor needs
to have the final say in the membership, because s/he needs to be able
to work well with all the board members.

Also, I have no idea how to define the community for the Independent
stream. ISOC members? RIR members? ICANN community? ACM? BCS? IEEE?
Where would it end?

    Brian

N.B. I am a member of the current RFC Editorial Board, by invitation
from the current RFC Editor.

---

From: John C Klensin <[email-masked]>
Date: Wed, 02 Sep 2009 18:33:02 -0400

--On Wednesday, September 02, 2009 13:53 -0400 Sam Hartman
<[email-masked]> wrote:

> John, in principle,  I would be delighted by this option if
> you made a few more changes to make the RFC process more
> accountable:
> 
> 1) Open up the rfc-editorial board so that it was selected by
> some sort of nomcom/community process.  That nomcom could of
> course draw from a broader community than the IETF as a whole
> 
> I have a lot more faith in the IETF process than I do the RFC
> editor process.  I believe that the RFC editor process is more
> open to a different type of abuse than the IESG process, but I
> believe we have a far more open process for addressing
> problems with the IESG than we do with IAB decisions about the
> RFC editor or with the RFC editor process itself.
> 

Sam,

Brian and Joel covered most of what I would have said had I
gotten to your note earlier.  I would add only three things to
their remarks:

(1) Checks and balances against what?  I trust the answer is not
"publication of high-quality articles which contain content with
which some AD happens to disagree" because that is the only
thing that has been at issue in the past.  On other matters, the
RFC Editor has _voluntarily_ deferred to the IESG, often going
well beyond what the current procedures require.  Remember that
RFC Editor acceptance of an IESG Note is voluntary today,
regardless of what the IESG might believe, _and_ that the
"written notice" procedure I suggested has been followed, as a
professional courtesy, for years, without any such requirement
being written down anywhere.   Put differently, what is the
threat model against which you are trying to defend?

(2) If I have to make a choice, I prefer to design systems to
deal with real and identified threats rather than purely
theoretical ones.  As I pointed out, we've had repeated examples
over the years of the IESG and/or individual ADs abusing the
independent submission process and/or the RFC Editor and zero
examples of the RFC Editor handling a request from the IESG
unreasonably or arbitrarily.   So why do you believe we need
more protection against the possibility of RFC Editor abuse than
we do against IESG abuse or, from a different perspective,
believe that the wider community would be better served by
tilting the balance even further toward the IESG?

(3) As Dave Crocker is fond of pointing out, the Nomcom cannot
be expected to make good appointments in areas that are outside
the expertise of most of its members... there is just no
foundation on which a Nomcom without that expertise can evaluate
candidates.  Given that, why do you believe that the Nomcom
could select an effective editorial board, with or without a
broader selection process?  Neither democracy nor randomness
seem to me to be guarantees of competence.   And, again, what
real problem do you think that would solve?

    john

---

From: <[email-masked]>
Date: Thu, 3 Sep 2009 09:04:22 +0200

John,

Your suggestion would largely address my concerns related to the
timely appeal path.

Best regards,
Pasi

> -----Original Message-----
> From: ext John C Klensin [mailto:[email-masked]]
> Sent: 02 September, 2009 20:20
> To: Eronen Pasi (Nokia-NRC/Helsinki); [email-masked];
> [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: RE: draft-housley-iesg-rfc3932bis and the optional/mandatory
> nature of IESG notes
> 
> 
>

---

From: Olaf Kolkman <[email-masked]>
Date: Thu, 3 Sep 2009 10:02:54 +0200

On Sep 2, 2009, at 7:20 PM, John C Klensin wrote:

> We simply

I think it is more than reasonable to expect documented communication  
if a request from the IESG is not honored by the ISE. I also think it  
is reasonable to expect such decision to be appealable/reviewable and  
that all the communication is timely.

So yes, I support this, in fact I take this for granted.

I believe that difference of technical opinion follows RFC 4846 sect 5  
(which is about publishing or not publishing). The denial of a note  
from the IESG is a dispute between streams that follows RFC 5620  
4.2.1. I can see that there could be some argument about which  
procedure to follow but in the end the RFC Editor (the RSE in this  
case) will make the final decisions based on various recommendations  
(that is how both 4846 and 5820 are written).

--Olaf (no hats)

--Olaf

________________________________________________________

Olaf M. Kolkman                        NLnet Labs
                                        Science Park 140,
http://www.nlnetlabs.nl/               1098 XG Amsterdam

---

From: Jari Arkko <[email-masked]>
Date: Thu, 03 Sep 2009 11:55:18 +0300

John,

> we've had repeated examples
> unreasonably or arbitrarily.

I don't want to open a discussion about who is more evil, particularly 
when opinions about any particular case probably differ. All I want to 
say about that is that as long as I have been looking, the score has 
been zero and zero on both sides. In particular, when I have been an AD 
it has always been a pleasure to work with the RFC Editor, and they have 
always made exactly the right decisions. In my honest opinion of course.

But I did want to bring up a couple of other angles. First of all, all 
the streams get their share of garbage. And sometimes the right decision 
is to publish the document despite it having some faults, or at least 
differences of opinion to established IETF practice. However, in such 
cases the notes that we are talking about really can be necessary (e.g., 
when a document redefines RADIUS Access-Reject as Access-Accept, to cite 
one real example from a few years back).

The second point was that in general, human organizations are prone to 
occasional failures. I at least prefer designs that are inherently 
capable of dealing with such failures (e.g., appeals path, way to fix a 
bad decision). However, I still want to see the RFC Editor as a simple 
journal-like function; please don't take my comment as an indication 
that board members should be selected by Nomcom, publication decisions 
should have public last calls or anything like that. We already have the 
IETF which runs in a community driven manner.

Jari

---

From: John C Klensin <[email-masked]>
Date: Thu, 03 Sep 2009 05:43:31 -0400

--On Thursday, September 03, 2009 11:55 +0300 Jari Arkko
<[email-masked]> wrote:

>...
> In particular, when I have been an AD it has always been a
> pleasure to work with the RFC Editor, and they have always
> made exactly the right decisions. In my honest opinion of
> course.

And I'd rather count on that and think about ways to deal with
any patterns of deviation if they should occur than start
devising ways for one stream to tell another one what it must do.
 
> But I did want to bring up a couple of other angles. First of
> all, all the streams get their share of garbage. And sometimes
> the right decision is to publish the document despite it
> having some faults, or at least differences of opinion to
> established IETF practice. However, in such cases the notes
> that we are talking about really can be necessary (e.g., when
> a document redefines RADIUS Access-Reject as Access-Accept, to
> cite one real example from a few years back).

And that is perhaps exactly an example for the other side.
First, it is at the very margin between the "procedural review
for conflicts with ongoing IETF work" that both 3932 and 4846
call for and the "technical review" that both documents more or
less indicate should be handled on an individual review basis.
I suggest that it is not so much a conflict with the ongoing
work of an IETF WG, but a flat technical error.  The right
solution for such errors is to fix them, or at least ask the
authors to include a detailed discussion of why the terminology
needs to be different, not a "note"... especially, IMO, one
that, per the current interpretations of the current RFC 3932,
is closer to "nah, nah, we don't like this" than a useful
explanation that identifies the specific problem.   I believe
that, if an ISE were to ignore reviews/input on a subject like
that, or fail to push the authors very hard after getting it, we
would have a very serious problem on our hands... whether that
review came from the IESG, from a single AD, from within some
relevant WG, or from a moderately random IETF participant or
onlooker.   

So I don't see that "notes" are the right solution to that
problem.  At worst, they become a crutch that reduces the ISE's
belief that it is the responsibility of the authors and the RFC
Editor function to actually get these documents right.

> The second point was that in general, human organizations are
> prone to occasional failures. I at least prefer designs that
> are inherently capable of dealing with such failures (e.g.,
> appeals path, way to fix a bad decision). 

We agree about that, which is one of the reasons why
institutionalizing the current practice of telling the IESG why
comments are being rejected (if that happens) strikes me as a
good idea.   But, to paraphrase (I think) Joel, I'll prefer to
see things work in a way that gets documents improved rather
than ones that drop cross-stream notes into them (mandatory or
not).  And that is especially so after many decades of
experience in watching people's eyes glaze over as they notice
what appears to be boilerplate and skip it... getting the text
right, or getting it to clearly note that there are dissenting
views and why, is just a much better way to get the point across
than note-inserting.

> However, I still
> want to see the RFC Editor as a simple journal-like function;
> please don't take my comment as an indication that board
> members should be selected by Nomcom, publication decisions
> should have public last calls or anything like that. We
> already have the IETF which runs in a community driven manner.

Good.  And thanks.
    john

---

From: Jari Arkko <[email-masked]>
Date: Thu, 03 Sep 2009 12:48:29 +0300

John,

> work of an IETF WG, but a flat technical error. 

And I would in general agree with you, but in this case the stuff was 
already deployed very widely, and the purpose of the publication was to 
document existing practice. I agreed that the draft had to be published 
as it was. Of course, a clean design would have been different, but what 
do you do?

Jari

---

From: John C Klensin <[email-masked]>
Date: Thu, 03 Sep 2009 16:12:28 -0400

--On Thursday, September 03, 2009 12:48 +0300 Jari Arkko
<[email-masked]> wrote:

> And I would in general agree with you, but in this case the
> stuff was already deployed very widely, and the purpose of the
> publication was to document existing practice. I agreed that
> the draft had to be published as it was. Of course, a clean
> design would have been different, but what do you do?

It is hard to argue against an accurate description of deployed
existing practice, even if it is conflicting or otherwise
terrible.  I think that the process should have caught the
terminology discrepancy and that the authors should have been
required to describe it, and the problems it could cause, very
carefully.

Again,  I think we are fairly closely aligned about the
principle here.  I just think that "notes" -- especially
non-specific ones that are, or appear to be, boilerplate-- are
rarely a desirable solution and that authors should be forced
into careful description of issues and/or critical reviews of
differences as a better alternative.  The job of doing the
"forcing" is that of the RFC Editor/ ISE, not the IESG, but I'd
be very concerned about any ISE who didn't take the
responsibility for ensuring that documents are not confusing wrt
IETF (and other identifiable) work were not clarified to the
point that a reader would have no trouble telling from context
was was going on.

It may amuse you to know that I've been pushing the theme that
independent submissions that even vaguely overlap with IETF work
should both show an awareness of that situation and include
critical reviews of the differences for many years, enough years
to have repeatedly made the comment to the previous RFC Editor
as well as the current one (and probably persistently enough to
occasionally irritate both).   I've also suggested from time to
time that, if someone can read an independent submission
(non-April-1) RFC that overlaps with IETF work with reasonable
care and not emerge with a clear understanding of the
relationship, the RFC Editor has not done their job.   I'm sort
of an extremist on the topic, even though the pragmatics of
getting useful documents posted have often caused me to not get
exactly what I've asked for.  But, again, I see the primary
responsibility for making sure that Independent submission
documents are clear about what they are lies 
with the authors and RFC Editor(ISE), that the considerable
improvements in Headers and Boilerplates will play a major role
in helping with that, and that qualifying "notes", regardless of
source, are a very poor choice, to be used cautiously,
selectively, with careful relationship to context, and only if
the issues cannot be resolved in any other way.

    john

---

From: Michael StJohns <[email-masked]>
Date: Thu, 03 Sep 2009 20:55:29 -0400

Hi John - 

I'm convinced we (the internet community) still need an true independent submissions path.  I'm no longer convinced that the path should or can lead through the RFC editor.

In the far past, the RFC Editor was a true independent entity - part of the internet community, participating and part of the IAB, but with its own funding source and a mandate to do the right thing.  The RFC Editor was a both a technical and stylistic reviewer and final arbiter of what got published - but that wasn't a very heavy burden for the community.

Over the years that independence has waned  - with the cessation of independent funding, with Jon Postel's death, with the termination of the IETF's CNRI relationship, and most recently with the competition of the RFC Editor function. Control has been centralized and the RFC Editor's editorial independence has all but been eliminated.  

At this point, it appears to be about who gets to decide.  And that goes back to the golden rule.  It's too easy for those who control the funding to control the publication, especially since the RFC editor function has mostly been reduced to stylistics without the ability (either contractually or technically) to act as a fair and independent decider. I may be overstating the case, but I can't see how it could be any different given what I've read in the solicitations.

I can't see any way to provide an objective set of publication rules that can be implemented by rote by the editor.  Which right now throws the subjective decisions over to the IESG which can have a conflict of interest with respect to certain submissions - hence the whole set of discussions about "notes".   

I fear the problem is intractable without reference to an editorial/publication decision function that is completely independent of the IESG. 

Mike