Subject: Call for Community Feedback: Retiring IETF FTP Service

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 02:23:58 +0000

Hi!

The Internet Engineering Steering Group (IESG) is seeking community input on retiring the IETF FTP service (ftp://ftp.ietf.org, ftp://ops.ietf.org, ftp://ietf.org).  A review of this service has found that FTP appears to serve a very small community and HTTP has become the access mechanism of choice.  Given this shift in community usage, reducing the operational complexity of the overall IETF infrastructure seems to outweigh the very limited community served with FTP.  

In reviewing the additional impacts of such a service retirement, the dependencies on FTP have been assessed.  Additionally, it has been confirmed that all information currently reachable through FTP will continue to be available through other services (HTTP, RSYNC, IMAP).

In consultation with the Tools team (Robert, Glen, Henrik, Russ, and Alexey), Communications team (Greg), affected SDO liaisons, IAB Chair, and LLC ED, a proposed retirement plan was developed and is available at:

https://www.ietf.org/media/documents/Retiring_IETF_FTP_Service.pdf

The IESG appreciates any input from the community on this proposal and will consider all input received by December 4, 2020 (to account for the upcoming IETF 109 and holidays).

Regards,
Roman
(as the IESG Tools Liaison)

---

From: George Michaelson <[email-masked]>
Date: Tue, 10 Nov 2020 12:38:19 +1000

What we have considered as part of our retirement option, is to ensure
the FQDN ftp.x.y.z continues to be live, but serves HTTP(S)

This means that URLs of the form ftp://ftp.x.y.z/path/to/data map
naturally to https://ftp.x.y.z/path/to/data

We have found this was helpful to remove the hits on FTP the protocol,
without massive impact to pURL burned into documents, or external
references we don't control.

It also leaves a trail, to confirm who comes in (by IP) on ftp, and
then who subsequently succeeds in fetching via HTTP(S). And, it
provides for redirect-permanent outcome.

cheers

-George

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 10 Nov 2020 19:26:21 +0900

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 10 Nov 2020 19:34:12 +0900

Sorry to have sent unedited mail.

Roman Danyliw wrote:

 > reducing the operational complexity of the overall IETF infrastructure

The best thing to do to reduce our operational effort is
to do nothing.

The second best thing is some clever and lazy officer just
declare to stop the service.

Are there any?

> In consultation with the Tools team (Robert, Glen, Henrik, Russ, and
> Alexey), Communications team (Greg), affected SDO liaisons, IAB
> Chair, and LLC ED, a proposed retirement plan was developed and is
> available at:

Wow! I'm impressed that there are so many hardworking guys.

						Masataka Ohta

---

From: "Scott O. Bradner" <[email-masked]>
Date: Tue, 10 Nov 2020 06:55:34 -0500

is there a compelling reason to stop a service that some people are using

the pdf says : "The operational complexity of running this service "

just what complexity is there once the service was set up (years ago)?

i.e., just how much does this service cost to run?
	(seems to me that it is likely that the effort to develop this plan was much more than just letting the service run)

yes, I run one of the scripts that use ftp to access IETF resources and it would be a significant pain to rewrite it
since it is complicated script and I do not know how to do some of its functions in other non-ftp ways

I do seriously want to know how much it costs the IETF to run the ftp service

Scott

> On Nov 9, 2020, at 9:23 PM, Roman Danyliw <[email-masked]> wrote:
> 
> Hi!
>

---

From: Lou Berger <[email-masked]>
Date: Tue, 10 Nov 2020 12:26:45 +0000

+1

Lou

----------
On November 10, 2020 6:56:04 AM "Scott O. Bradner" <[email-masked]> wrote:

Scott

Hi!

Regards,
Roman

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 10 Nov 2020 12:38:43 +0000

+1

Stewart

> On 10 Nov 2020, at 11:55, Scott O. Bradner <[email-masked]> wrote:
>

---

From: John C Klensin <[email-masked]>
Date: Tue, 10 Nov 2020 08:02:27 -0500

+1

And, while I suspect my scripts are less complicated than
Scott's, I do have them and am dependent on them.   So two
additional thoughts:

(i) I know the conventional wisdom in the IETF is to obsolete
HTTP in favor of HTTPS.  However, if conversation is necessary,
conversion from FTP to simple, no negotiation HTTP is likely to
be lots easier the conversation to HTTPS, certificate handling,
etc.  So, while the report seems to circle around this a bit, if
FTP is discontinued, will we be assured that plain HTTP access
will be available long-term rather than those who do convert
waking up one day and discovering that HTTP is being
discontinued because HTTPS is more virtuous?

(ii) Can we start evaluating these changes, not just in terms of
extra facilities that have to be maintained (and I agree with
Scott that, once the machinery is set up, the marginal costs of
maintaining FTP should be, well, marginal) but in terms of costs
to the community of discussing and making the conversion.
Except for those who actually have unlimited time or would not
be contributing to the IETF's technical work anyway and In
addition to the time taken up creating plans and reports like
this, please assume that every minute taking up considering a
plan like this, converting scripts (or even old habits), etc.,
is a minute that IETF technical, standards-producing, work isn't
getting done.  Is it worth it for this case (or any particular
other one that might arise next)?

     john

--On Tuesday, November 10, 2020 06:55 -0500 "Scott O. Bradner"
<[email-masked]> wrote:

> 	(seems to me that it is likely that the effort to develop
> this plan was much more than just letting the service run)
> 
> yes, I run one of the scripts that use ftp to access IETF
> resources and it would be a significant pain to rewrite it
> since it is complicated script and I do not know how to do
> some of its functions in other non-ftp ways
> 
>> The Internet Engineering Steering Group (IESG) is seeking
>> community input on retiring the IETF FTP service
>> (ftp://ftp.ietf.org, ftp://ops.ietf.org, ftp://ietf.org).  A
>> review of this service has found that FTP appears to serve a
>> very small community and HTTP has become the access mechanism
>> of choice.  Given this shift in community usage, reducing the
>> operational complexity of the overall IETF infrastructure
>> seems to outweigh the very limited community served with FTP.
>> 
>> 
>> In reviewing the additional impacts of such a service
>> retirement, the dependencies on FTP have been assessed.
>> Additionally, it has been confirmed that all information
>> currently reachable through FTP will continue to be available
>> through other services (HTTP, RSYNC, IMAP).
>> 
>> In consultation with the Tools team (Robert, Glen, Henrik,
>> Russ, and Alexey), Communications team (Greg), affected SDO
>> liaisons, IAB Chair, and LLC ED, a proposed retirement plan
>> was developed and is available at:
>> 
>> The IESG appreciates any input from the community on this
>> proposal and will consider all input received by December 4,
>> 2020 (to account for the upcoming IETF 109 and holidays).
>> 
>> Regards,
>> Roman

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 14:02:59 +0000

Hi George!

> -----Original Message-----
> From: George Michaelson <[email-masked]>
> Sent: Monday, November 9, 2020 9:38 PM
> To: Roman Danyliw <[email-masked]>
> Cc: [email-masked]
> Subject: Re: Call for Community Feedback: Retiring IETF FTP Service
> 
> What we have considered as part of our retirement option, is to ensure the
> FQDN ftp.x.y.z continues to be live, but serves HTTP(S)
> 
> This means that URLs of the form ftp://ftp.x.y.z/path/to/data map naturally to
> https://ftp.x.y.z/path/to/data

Indeed that's largely the thinking.  Additionally, the HTTP version of all of these mappings are already live (and have been for a number of years).  Currently, everything that was:

ftp://*.ietf.org is (and will continue to be) at http{s}://www.ietf.org/ietf-ftp/

Additionally,
ftp://*.ietf.org/rfc/* ==> http{s}://www.ietf.org/rfc/*
ftp://*.ietf.org/internet-drafts/ ==> http{s}://www.ietf.org/internet-drafts/*

Regards,
Roman

> We have found this was helpful to remove the hits on FTP the protocol, without
> massive impact to pURL burned into documents, or external references we
> don't control.
> 
> It also leaves a trail, to confirm who comes in (by IP) on ftp, and then who
> subsequently succeeds in fetching via HTTP(S). And, it provides for redirect-
> permanent outcome.
> 
> cheers
> 
> -George
> 
> > The Internet Engineering Steering Group (IESG) is seeking community input on
> retiring the IETF FTP service (ftp://ftp.ietf.org, ftp://ops.ietf.org, ftp://ietf.org).
> A review of this service has found that FTP appears to serve a very small
> community and HTTP has become the access mechanism of choice.  Given this
> shift in community usage, reducing the operational complexity of the overall
> IETF infrastructure seems to outweigh the very limited community served with
> FTP.
> >
> > In reviewing the additional impacts of such a service retirement, the
> dependencies on FTP have been assessed.  Additionally, it has been confirmed
> that all information currently reachable through FTP will continue to be
> available through other services (HTTP, RSYNC, IMAP).
> >
> > In consultation with the Tools team (Robert, Glen, Henrik, Russ, and Alexey),
> Communications team (Greg), affected SDO liaisons, IAB Chair, and LLC ED, a
> proposed retirement plan was developed and is available at:
> >
> > The IESG appreciates any input from the community on this proposal and will
> consider all input received by December 4, 2020 (to account for the upcoming
> IETF 109 and holidays).
> >
> > Regards,
> > Roman

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 14:33:38 +0000

Hi!

> From: ietf <[email-masked]> On Behalf Of Masataka Ohta
> Sent: Tuesday, November 10, 2020 5:34 AM
> To: [email-masked]
> The best thing to do to reduce our operational effort is to do nothing.

Noted, you would like the FTP service to continue.

I don't follow on how doing nothing reduces operational effort -- ongoing maintenance is always needed.  Nevertheless, doing nothing is not possible.  The underlying storage holding all IETF content is being refreshed.  Positive action is required to migrate all services which created a natural inflection point to consult the community on FTP based on observed usage patterns.

> The second best thing is some clever and lazy officer just declare to stop the
> service.

You are welcome to provide feedback that you don't think we should retire the service.  However, you are crossing the line with your personal attack of calling me lazy.

You correctly note that I serve on the IESG, a NomCom eligible position.  If you have strong concerns about my performance, I have put name in to be considered for another term as Security AD.  Please share your concerns with the NomCom at https://datatracker.ietf.org/nomcom/2020/ (clicking on the "provide feedback" link).
 
> > Alexey), Communications team (Greg), affected SDO liaisons, IAB Chair,
> > and LLC ED, a proposed retirement plan was developed and is available
> > at:
> 

Correct, there a number of hard wording men and women doing their best to serve the community.  Thanks for acknowledging them.

The various members of the tools team provided and helped with log analysis.  The communications team reviewed the web site content for dependencies.  A few of the RFCs found to have ftp pointers documented activities with other SDOs so the IETF liaisons (to IEEE and ITU-T) provided guidance on how to potentially handle that if we proceed with this plan.  These documents were in the IAB stream, so the IAB chair helpfully provided a review.  The LLC Executive Director also provided early feedback on a draft.

Regards,
Roman

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 10 Nov 2020 23:47:48 +0900

> Noted, you would like the FTP service to continue.

Wrong. I'm saying we should not be bothered by discussions to
continue or discontinue the service.

 > Correct, there a number of hard wording men and women doing their
 > best to serve the community.  Thanks for acknowledging them.

I thank all of you, very much.

						Masataka Ohta

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 14:47:46 +0000

Hi Scott!

> From: ietf <[email-masked]> On Behalf Of Scott O. Bradner
> Sent: Tuesday, November 10, 2020 6:56 AM
> To: [email-masked]
> Cc: [email-masked]

The complexity is having the "n+1" service to run.

> 	(seems to me that it is likely that the effort to develop this plan was
> much more than just letting the service run)
> 
> yes, I run one of the scripts that use ftp to access IETF resources and it would be
> a significant pain to rewrite it since it is complicated script and I do not know 
> how to do some of its functions in other non-ftp ways
>

As far as I know, there are no "billable hours" kinds of numbers on what it takes to run FTP.  As noted in an earlier response, the underlying storage holding all IETF content is being refreshed.  Positive action is required to migrate all services which created a natural inflection point to consult the community on FTP based on the observed usage patterns.  It appears the using HTTP is overwhelming more popular (details in the proposal).  I acknowledge that this isn't your workflow and hear that you'd like the FTP service to continue to support your scripts.

You noted that your scripts perform functions that you don't know how to do in "non-ftp ways".  Can you clarify that use case/function more (offline ok too)?   In preparing the contingency plan, it appeared that using an HTTP API or rsync covered the known use cases.

Regards,
Roman

Roman

---

From: "John Levine" <[email-masked]>
Date: 10 Nov 2020 11:54:16 -0500

In article <[email-masked]> you write:
>You noted that your scripts perform functions that you don't know how to do in "non-ftp ways".  Can you
>clarify that use case/function more (offline ok too)?   In preparing the contingency plan, it appeared
>that using an HTTP API or rsync covered the known use cases.

I do all of my scripted downloads with wget or curl which I think is
pretty typical. In those two programs, changing from ftp to https generally
only involves changing ftp:// to https:// in the command arguments.

I'd also be interested in hearing about what other FTP clients people are using
that are hard to switch to https.

R's,
John

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 17:19:01 +0000

Hi John!

> From: ietf <[email-masked]> On Behalf Of John C Klensin
> Sent: Tuesday, November 10, 2020 8:02 AM
> To: Scott O. Bradner <[email-masked]>; [email-masked]
> Cc: [email-masked]
> (i) I know the conventional wisdom in the IETF is to obsolete HTTP in favor of
> HTTPS.  However, if conversation is necessary, conversion from FTP to simple,
> no negotiation HTTP is likely to be lots easier the conversation to HTTPS,
> certificate handling, etc.  So, while the report seems to circle around this a bit,
> if FTP is discontinued, will we be assured that plain HTTP access will be
> available long-term rather than those who do convert waking up one day and
> discovering that HTTP is being discontinued because HTTPS is more virtuous?

To be clear, the proposal is completely reductive -- spinning down FTP.  The posture of HTTP vs. HTTPs is outside the scope of this proposal and would be a separate community discussion to change that (and I'm not aware of this being under consideration).

> (ii) Can we start evaluating these changes, not just in terms of extra facilities
> that have to be maintained (and I agree with Scott that, once the machinery is
> set up, the marginal costs of maintaining FTP should be, well, marginal) but in
> terms of costs to the community of discussing 

There is a tension here.  The community has made a clear it wants to be consulted (and it seems very appropriate to asked when taking something away).  However, reviewing this consultation takes time and effort from the community.  I'm not sure how to reconcile this other than erring on the side of asking when the issue come up (as has been done here).

> and making the conversion.

I acknowledge that this conversion cost is real.  However, so is the cost migrating and managing the filesystem.  It's also not entirely clear on how to reconcile these costs.  All I believe we have is the usage data which suggests that most (but not all) of the community prefers HTTP.  This strongly suggests that the aggregate workflow for the overwhelming majority of the community is using HTTP and that continuing to run FTP is to support a small fraction of the community.  This call for feedback is a solicitation to the community to help reason about this threshold of usage for a service.

> Except for those who actually have unlimited time or would not be contributing
> to the IETF's technical work anyway and In addition to the time taken up
> creating plans and reports like this, please assume that every minute taking up
> considering a plan like this, converting scripts (or even old habits), etc., is a
> minute that IETF technical, standards-producing, work isn't getting done.  Is it
> worth it for this case (or any particular other one that might arise next)?

Philosophically, this cuts both ways.  For every minute we are running a service that support n-users, we could be using that time to optimize a service that support 100n - 1000n-users.  That's the order of magnitude usage ratios were are talking about.  As noted in the proposal, this is likely undercounting HTTP usage since datatracker and cloudflare were not used.

Roman

> > 	(seems to me that it is likely that the effort to develop this plan
> > was much more than just letting the service run)
> >
> > yes, I run one of the scripts that use ftp to access IETF resources
> > and it would be a significant pain to rewrite it since it is
> > complicated script and I do not know how to do some of its functions
> > in other non-ftp ways
> >
> >> The Internet Engineering Steering Group (IESG) is seeking community
> >> input on retiring the IETF FTP service (ftp://ftp.ietf.org,
> >> ftp://ops.ietf.org, ftp://ietf.org).  A review of this service has
> >> found that FTP appears to serve a very small community and HTTP has
> >> become the access mechanism of choice.  Given this shift in community
> >> usage, reducing the operational complexity of the overall IETF
> >> infrastructure seems to outweigh the very limited community served
> >> with FTP.
> >>
> >>
> >> dependencies on FTP have been assessed.
> >> Additionally, it has been confirmed that all information currently
> >> reachable through FTP will continue to be available through other
> >> services (HTTP, RSYNC, IMAP).
> >>
> >> The IESG appreciates any input from the community on this proposal
> >> and will consider all input received by December 4,

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 10 Nov 2020 17:25:18 +0000

Hello!

> From: Masataka Ohta <[email-masked]>
> Sent: Tuesday, November 10, 2020 9:48 AM
> To: Roman Danyliw <[email-masked]>; [email-masked]
> Wrong. I'm saying we should not be bothered by discussions to continue or
> discontinue the service.

My understanding is that the community likes to be consulted about changes in the services it gets which is why it is being asked.

Roman

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Tue, 10 Nov 2020 09:30:17 -0800

Transferring files that are not HyperText files using HTTP is in poor taste, much like eating salad with a fork that is not a salad fork.

-- 
Marc Petit-Huguenin
Email: [email-masked]
Blog: https://marc.petit-huguenin.org
Profile: https://www.linkedin.com/in/petithug

---

From: "John Levine" <[email-masked]>
Date: 10 Nov 2020 12:53:03 -0500

>a fork that is not a salad fork.

Better tell Tim B-L.  His HTTP/1.0 RFC in 1996 allowed arbitrary MIME types.

I also think these days you may find the occasional web page with
embedded images and other media, all of which are served over http.

R's,
John

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 10 Nov 2020 18:15:03 +0000

>    Transferring files that are not HyperText files using HTTP is in poor taste, much like eating salad with a fork that is not a salad fork.

Strongly disagree.

Particularly when HTTPS is used, you get privacy and an *authenticated connection* to the server.  FTP doesn't have that, and while SFTP or FTPS do, they are not trivial to set up.

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 10 Nov 2020 23:55:02 +0100

a) I don't think that "small community" is a sufficient analysis to
   justify retirement of ftp. If this small community for example would be
   significant mirrors or give access to the content to users who for 
   whatever reasons can not access the content differently, i am sure that
   would influence the decision making.

b) I don't know what "operational complexity" is supposed to mean.
   Cost of operations would be a more useful measure to weigh against 
   the benefit, especially when its hard to quantify the benefit, given
   how (see a) the benefit is not necessarily equal to the percentage of
   utilization - and the cost likely is very small.

c) The PDF and your email is not really honest, because it says "http",
   but in reality every http URL is immediately redirected to https, aka:
   retiring ftp would further strengthen the policy USER MUST USE END
   TO END ENCRYPTION WHETHER THEY WANT TO OR NOT.

   I understand that this is the implied policy by security advocacy in the
   IETF, but i think it is not the right policy for all content. I think
   end-to-end encryption should be a choice. Of course, a content provider
   like IETF could make that choice and force it upon users, but for example
   for bulk download of public data it often is a quite performance reducing
   choice.

   Aka: https downloads are slower than ftp/rsync (rsync without ssh!).
   Encrypted download might also not be permitted by users operating
   from secured environments where all actions need to be logged. Aka:
   I think IETF should have the option for unencrypted access to its
   data for all popular protocols and leave the choice to the users.

   Of course, this would be less a question of protocol if IETF would not
   have forced MUST ENCRYPT onto TLS 1.3, because no-encrypt,authenticate-only
   would be a perfect option for these type of use-cases when accessing public
   data without requring privacy. Oh well...

d) HTTP/HTTPS are not protocols that actually allow to search/find content,
   like FTP is with its ability to list directories. For repositories like what
   was/is on ftp, HTTP/HTTPS are severely inferior protocols. Directory listing
   through per-client implemented screen scrping of directory listing output
   format of few well-known web-server directory listing formats. And IETF
   seems to be endorsing this. Give me a break. Where is INT area when you
   need it ? ;-)) Do we even specify a standard format for IETF HTTP/HTTP
   directory listing format, or do all mirroring script clients need to be
   updated when/if the IETF web server would some time decide to change
   implementations and use a different directory listing output format ?

   WebDAV ? Have not seen this in operation on any browser to give me
   better, FTP like experience on web file stores...

e) rsync would survive as the only viable bulk update mechanism, and for
   that one may argue its better than FTP. I am a fan. But not sure
   that means it can replace ftp in all use-cases.

   The IETF documentation for rsync
   is quite lacking. I would stroongly suggest to put better documentation,
   on a https://rsync.tools.ietf.org page so users for whom https is not the
   best choice can quickly get up to speed.

   Also: before removing ftp.ietf.org, i would STRONGLY suggest to simply
   remove all content, but keep the web service alive with a forward pointer,
   (README) e.g.: to http://rsync.tools.ietf.org - and let that run for at least
   one year (or longer). That way, all owners of scripts could inf the
   forward information.

f) Q: Was this type of question raised before investing all the effort
   into the analysis of the situation and what looks like almost finished
   decision making ? i can't remember and earlier Q to the community...

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 11 Nov 2020 00:11:51 +0100

> Particularly when HTTPS is used, you get privacy and an *authenticated connection* to the server.  FTP doesn't have that, and while SFTP or FTPS do, they are not trivial to set up.

How does this make FTP the better salad fork ? 

I may not want encryption because it reduces my bulk download speed,
yet ietf does not even give me http download option but forces https.

My client needs to do directory screen scraping to know whats on the
server with HTTP.

IETF content is public. I may not even want/need to download it from
*.ietf.org, i may prefer a faster mirror:

Its a pretty bad "Inernet" security design to expect authenticity of content
 by having authenticity of transport. That type of strict 1:1 authenticity
maping better suits walled garden solutions.

IMHO: all IETF documents should better have an IETF/RFC-editor signature
on them, like we do for S/MIME (not sure if there are good standards
for such document signatures for the most important docs: XML, Text,
email(box).

Cheers
    Toerless

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 11 Nov 2020 21:19:29 +0900

Toerless Eckert wrote:

> c) The PDF and your email is not really honest, because it says
> "http", but in reality every http URL is immediately redirected to
> https, aka: retiring ftp would further strengthen the policy USER
> MUST USE END TO END ENCRYPTION WHETHER THEY WANT TO OR NOT.

https, with intelligent intermediate entities of CAs, or PKIs
in general, are not secure end to end vulnerable to MitM
attacks on the CAs.

Despite my warning here, which was privately denied by an
security AD without reasoning, a CA of Diginotar was actually
compromised to attack google successfully.

So, never say a PKI, including https, is secure end to end.

					Masataka Ohta

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 11 Nov 2020 14:33:03 +0000

Hi John!

> From: Roman Danyliw
> Sent: Tuesday, November 10, 2020 12:19 PM
> To: 'John C Klensin' <[email-masked]>; Scott O. Bradner <[email-masked]>;
> [email-masked]
> Cc: [email-masked]
> Subject: RE: Call for Community Feedback: Retiring IETF FTP Service
> 
> Hi John!
> 
> > (i) I know the conventional wisdom in the IETF is to obsolete HTTP in
> > favor of HTTPS.  However, if conversation is necessary, conversion
> > from FTP to simple, no negotiation HTTP is likely to be lots easier
> > the conversation to HTTPS, certificate handling, etc.  So, while the
> > report seems to circle around this a bit, if FTP is discontinued, will
> > we be assured that plain HTTP access will be available long-term
> > rather than those who do convert waking up one day and discovering that
> HTTP is being discontinued because HTTPS is more virtuous?
> 
> To be clear, the proposal is completely reductive -- spinning down FTP.  The
> posture of HTTP vs. HTTPs is outside the scope of this proposal and would be a
> separate community discussion to change that (and I'm not aware of this being
> under consideration).

I was privately reminded of a particular nuance that I wanted to make note of publicly -- IETF web properties (www.ietf.org, datracker.ietf.org, etc.) are all already HTTPS only, and has been the case for several years.

Regards,
Roman

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 11 Nov 2020 14:56:56 +0000

Hi John!

> Sent: Wednesday, November 11, 2020 9:33 AM
> To: 'John C Klensin' <[email-masked]>; 'Scott O. Bradner' <[email-masked]>;
> [email-masked]' <[email-masked]>
> Cc: [email-masked]' <[email-masked]>
> > To: 'John C Klensin' <[email-masked]>; Scott O. Bradner
> > <[email-masked]>; [email-masked]
> > Cc: [email-masked]
> > > (i) I know the conventional wisdom in the IETF is to obsolete HTTP
> > > in favor of HTTPS.  However, if conversation is necessary,
> > > conversion from FTP to simple, no negotiation HTTP is likely to be
> > > lots easier the conversation to HTTPS, certificate handling, etc.
> > > So, while the report seems to circle around this a bit, if FTP is
> > > discontinued, will we be assured that plain HTTP access will be
> > > available long-term rather than those who do convert waking up one
> > > day and discovering that
> > To be clear, the proposal is completely reductive -- spinning down
> > FTP.  The posture of HTTP vs. HTTPs is outside the scope of this
> > proposal and would be a separate community discussion to change that
> > (and I'm not aware of this being under consideration).
> 
> I was privately reminded of a particular nuance that I wanted to make note of
> publicly -- IETF web properties (www.ietf.org, datracker.ietf.org, etc.) are all
> already HTTPS only, and has been the case for several years.

And to clarify further, "several years" ago was 2015 per https://www.ietf.org/about/groups/iesg/statements/maximizing-encrypted-access/

Roman

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 12 Nov 2020 02:57:35 +0000

Hi!

I've received various private inquiries for additional clarity on FTP usage and what constitutes "a small community" beyond what's already in the proposed retirement plan [1].  To that end, I pulled together a few more detailed stats.  This more detailed survey only looks at one of the periods referenced in [1] so the stats will vary a little bit (but the trends are the same).  Please excuse the lack of polish and document location as this was quick and dirty.  See:

https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJsHI4uyDlQ2dHip8jo/edit?usp=sharing

A few summary observations:
** ~91 users of FTP (from 140 unique IPs)
** 78% of all traffic created by 5 users
** 40% of FTP users made 1 file request
** FTP is 0.2% of HTTP traffic (to documents)
** >99% of all FTP traffic is users performing bulk downloads or search engine behavior

Regards,
Roman

[1] https://www.ietf.org/media/documents/Retiring_IETF_FTP_Service.pdf

> Sent: Monday, November 9, 2020 9:24 PM
> To: [email-masked]' <[email-masked]>
> Subject: Call for Community Feedback: Retiring IETF FTP Service
> 
> Hi!
>

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 12 Nov 2020 03:40:34 +0000

Hi Toerless!

Thanks for the detailed feedback.

> From: Toerless Eckert <[email-masked]>
> Sent: Tuesday, November 10, 2020 5:55 PM

Absolutely, if we were convinced that FTP provided unique access that might change the calculus.  However, this seems like a hypothetical not backed by the data and infrastructure we have right now.

As noted in the proposal, ALL data is available at least two other ways (https://www.ietf.org/ietf-ftp or rsync).  By request volume, HTTPS is massively preferred (FTP is 0.2% of HTTPS document traffic, and this is undercounting HTTPS usage) by the vast majority of users.  If the bulk download semantics are desired, to include incremental updates with no code required, then rsync is the best choice.  Do you have a user community in mind that can use FTP, but not HTTPS or rsync that we need to consider?

Per the usage data [1], the 85th percentile of traffic comes from entities that don't strongly suggest they would mirror for unique access:

** a dynamic IP address in a German ISP
** the proxy of a Fortune 100 company
** a Canadian IT services company
** a large US search engine company
** a leading Japanese research university ++
** website of a not-so-popular programming language
** a small Swedish software product company
** a small, several person US consulting company

++ maybe this one, but a cursory look didn't find an FTP server making that content available or a mirror of it served by HTTP 

>    utilization - and the cost likely is very small.

Running each services has a cost. In the case of FTP, there is of course the FTP service itself.  On the backend, to make all of the data replicate in such a way that exposes it correctly to FTP requires custom tooling that must be maintained.

You are exactly right about the benefit being a key consideration.  Usage appears to be extremely low which is why the community is being consulted.  That said, the usage is not zero.

I won't repeat the usage summary from the proposal or [1].
 
>    for bulk download of public data it often is a quite performance reducing

Noted.  The text could have been more precise as HTTPS.  I'd prefer not to relitigate what was decided 5 years ago (https://www.ietf.org/about/groups/iesg/statements/maximizing-encrypted-access/).

>    Aka: https downloads are slower than ftp/rsync (rsync without ssh!).

Given that >99% of all traffic is bulk downloads [1] and in the grand scheme of things pretty low volumes, can you clarify the use case in which the performance difference between ftp vs. https practically matters.

>    data for all popular protocols and leave the choice to the users.

Can you say more about these secured environments?  Those that I am familiar with would not allow insecure protocols like FTP, would be air-gapped not allowing any internet access, or would heavily filter what sites on the internet with which there is contact (and ietf.org doesn't seem like one to allow)

>From the data we have [1], it doesn't seem like many users are operating in a restricted environment.  I'm making assumptions here, but the biggest users of FTP, constituting 85% of the traffic, all seem very capable of consuming encrypted content.
** a dynamic IP address in a German ISP
** the proxy of a Fortune 100 company
** a Canadian IT services company
** a large US search engine company
** a leading Japanese research university
** website of a not-so-popular programming language
** a small Swedish software product company
** a small, several person US consulting company

I stopped counting at 38, but that's the number of addresses in a dynamic IP address range of a North American or European ISP, or at a university.  Those seem like safe bets for having unrestricted HTTPS access to reach ietf.org.  Combine that 38 + the 8 above gets one reasonable confident that at least half of the 91 users aren't in these restricted environments.  The numbers likely much higher.

Given the number of replicas on the internet of the I-Ds and RFCs, if there were many such disadvantaged users, couldn't they go there?

>    better, FTP like experience on web file stores...

No argument on your observation.  

>From the data we have in [1], there is little evidence of "search/find" happening, if at all on our FTP service.  Consider the more detailed analysis as [1].

"Search/find activity" would likely not look like the access patterns of:
** Search engine
** Sync of the repo
** Incremental sync of the repo
** Polling a single file
** Polling the status files
** Single file access

This leaves only the "unknown".  Practically that is 12 users who collectively made 60 files access.  Two of those users downloaded the same document twice, to me that doesn't say discovery.  That leaves 10 users with 58 files accesses:

** user 1: 11 file accesses
** user 2 - 4: 8 file accesses
** user 5: 5 file accesses
** user 6-7: 4 file accesses
** user 8-9: 3 file accesses
** user 10: 2 file accesses

This is pretty light "search and find" usage if that is it at all.  For perspective, this is 11% of FTP users, 0.1% of all FTP traffic and 0.001% of the traffic compared to HTTPS.

>    that means it can replace ftp in all use-cases.

Completely agree.  It appears from the analysis that at least 86% (from [1], "full sync" and "incremental sync") of all requests could be satisfied by rsync.  Turning off FTP will also eliminate the 12% of requests that come from search engine traffic.  This leaves the use cases of the following, which if scripted would require code changes:

Polling single file (0.1% of FTP traffic) -- ready means to implement with HTTPS

Polling status file(s) (0.3% of FTP traffic) -- ready means to implement with HTTPS

Single file access (0.1% of FTP traffic, 37 requests) --  don't understand enough to say, perhaps inbound links for a ftp:// which can't be changed

Unknown (0.2% of FTP traffic, 60 requests) -- don't understand enough to say

>    best choice can quickly get up to speed.

Point taken.  In fairness, ~86% of these requests (i.e., those that sync I-Ds and RFCs) would be handled by this documentations which covers rsync:

https://www.ietf.org/standards/ids/internet-draft-mirror-sites/

>    one year (or longer). That way, all owners of scripts could inf the

Noted as an additional retirement transition approach.  From everything I've heard from two script owners who have spoken up, this unfortunately wouldn't help them.  Although using rsync might.

>    decision making ? i can't remember and earlier Q to the community...

Nothing is finished.  This more detailed plan was prepared in response to many of the questions left unanswered in the 2015 discussion -- how much is FTP being used?  Is access to information going to be lost? what about stable references in documents?  Anticipating these same questions would be asked, this plan was prepared.

Regards,
Roman

[1] https://mailarchive.ietf.org/arch/msg/ietf/bEIgcOFMA73s5BG_vZppQsUZ2eU/

---

From: Larry Masinter <[email-masked]>
Date: Wed, 11 Nov 2020 21:45:11 -0800

I read your data differently. If' you have ~91 different users that's a lot.
I agree with the sentiment that you've spent more attention bandwidth
trying to retire the service than it should take to run it a few more years.
Chrome had a 2-year plan
https://docs.google.com/document/d/1JUra5HnsbR_xmtQctkb2iVxRPuhPWhMB5M_zpbuGxTY/edit#heading=h.7bs9792ml0ln
to remove ftp from the browser by 2q 2021

  https://news.ycombinator.com/item?id=20721609
Anyway, cool url's don't change.

but c

[2]

---

From: Loa Andersson <[email-masked]>
Date: Thu, 12 Nov 2020 15:08:54 +0800

All,

That was my thought also, if we claim "operational complexity" we should 
maybe try to be explicit.

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Bronze Dragon Consulting             phone: +46 739 81 21 64

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 13 Nov 2020 15:44:28 +0000

Hi Larry!

Per “~91 different users that's a lot”, I’d like to put that into perspective – this number means that 140 unique IP address used the FTP service.  In comparison, _only one_ of our HTTPS services exposing only a subset of this information saw on average ~654,430 unique addresses.  This makes FTP at best 0.02% of unique visitors (again, likely a lot less if full HTTPS logs were available).

I don’t mean to trivialize the visits of these 140 IPs (minus the 35 that came from a search engine looking for robots.txt), but let’s be clear, there is a very, very, very small population being served with FTP as measured by request volume (<0.2%) or unique visitors (<0.02%).  See [1] for details.  No access to information is being taken away.  As noted in [2], alternative means for access the same information is available.  Furthermore, analysis of the usage patterns [1] suggests that most of the requests (96-98%) are trying to sync parts of the FTP repo.  This style of access can be replicated with rsync.  Hence, not only is access preserved, but also possible without custom tooling.

Bottom line (for me) is that the data overwhelming shows that the community has moved to using other services.  This is why the Tools Team asked the IESG to pose the question to the community.

I fully recognize that there may be custom scripts among those IP addresses hitting FTP.  It isn’t clear how many, but it is no more than half of the 140 IP addresses [1].  Sunsetting FTP will break some code – those users will incur a penalty here.

To the community -- If you can’t or don’t have time to convert your FTP script; or need help configuring rsync, please reach out directly to me.  I, volunteer developers from my employer (Carnegie Mellon University’s Software Engineering Institute) or the Tools Team, within reason, will help you port your code or workflow, or answer questions about the services provided by the IETF infrastructure.

Lastly, I’ve been sharing all sorts of updated stats in response to questions.  I want to acknowledge and thank Glen Barney, Henrik Levkowetz, and Robert Sparks on the Tools Team who are pulling these numbers for us (found in [1] and [2]) and clarifying infrastructure details so we’re not speculating about hypotheticals.

Regards,
Roman

[1] https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJsHI4uyDlQ2dHip8jo/edit#
[2] https://www.ietf.org/media/documents/Retiring_IETF_FTP_Service.pdf
[3] https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5dAb3FAgNng/

From: Larry Masinter <[email-masked]>
Sent: Thursday, November 12, 2020 12:45 AM
To: Roman Danyliw <[email-masked]>
Cc: Toerless Eckert <[email-masked]>; [email-masked]
Subject: Re: Call for Community Feedback: Retiring IETF FTP Service

I read your data differently. If you have ~91 different users that's a lot.

[Roman] That is for the community to discuss.

Speaking personally, at first blush, perhaps 91 might seem like a lot.

We might have to agree to disagree on this.  On the face it, perhaps 91 is a lot,

I agree with the sentiment that you've spent more attention bandwidth trying to retire the service than it should take to run it a few more years.
Chrome had a 2-year plan
https://docs.google.com/document/d/1JUra5HnsbR_xmtQctkb2iVxRPuhPWhMB5M_zpbuGxTY/edit#heading=h.7bs9792ml0ln
to remove ftp from the browser by 2q 2021

but c

[2]

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Nov 2020 12:00:16 -0500

I'm very opposed to this proposal.

FTP is a much better interface than HTTP for scripting, mirroring, and 
remote file access (i.e. mounting an FTP server like a "share" so that 
it can be accessed from one's computer just like any other file system)

FTP is also a very stable interface, whereas as HTTP + HTML are both 
hideously complex.   Use of FTP as a remote file access protocol is very 
widely supported.

Basically what makes an effective human interface, and what makes a good 
interface for computer programs, are very different. HTTP+HTML is 
arguably an example of the former (though I think it still has a long 
way to go and needs to evolve toward less complexity rather than more 
complexity); FTP is better at the latter.

But IETF should fundamentally be about promoting stable interfaces.   
That is, after all, the purpose of standards.

More deeply, I think this proposal represents a fundamental 
misunderstanding about what IETF is, or should be, about.   We need to 
be facilitating tool building, not trying to stamp it out by forcing 
tool builders to use deeply flawed APIs.   The idea that RFCs are only 
supposed to be accessed through web browsers is incredibly backward.

Also, *don't interpret traffic volume as an indicator of importance*.   
Sure, if nobody ever used FTP at all, it would probably be safe to say 
it's not important.   But a small amount of usage can still be of 
tremendous benefit to the community, and HTTP simply doesn't work as 
well (in practice) for that.

Keith

p.s. now having said the above, I will admit that in the past my scripts 
have always used the RFC editor's FTP server rather than the IETF's - 
why not go to the authoritative source of the information?   But I also 
have doubts about the RFC Editor continuing to provide what the 
community needs, as the separation of interests between the RFC Editor 
and the IETF seems to be growing over time, and the RFC Editor has 
undermined the IETF on at least one occasion.   On balance, I hope the 
IETF continues to serve as an alternate trustworthy source of RFCs, and 
also that the IETF continues to make RFCs available via both effective 
human interfaces and effective, stable machine-accessible APIs.

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Nov 2020 12:07:02 -0500

> As noted in the proposal, ALL data is available at least two other ways (https://www.ietf.org/ietf-ftp  or rsync).  By request volume, HTTPS is massively preferred (FTP is 0.2% of HTTPS document tr…

rsync is not nearly as widely supported as FTP.   And while rsync works 
okay for mirroring, I've never seen it used for remote file access.

Really this proposal is a tremendously bad idea, and it should be withdrawn.

Keith

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 13 Nov 2020 12:43:02 -0500

On Fri, Nov 13, 2020 at 12:01 PM Keith Moore <[email-masked]>
wrote:

> remote file access (i.e. mounting an FTP server like a "share" so that it
> can be accessed from one's computer just like any other file system
>
I disagree. The ability to mount HTTP file systems is actually built into
Windows. I don't think FTP is supported.

FTP is a very peculiar protocol, I have implemented it several times and it
was awful to do even before NATs got in the way. It is not really a
separate protocol, it is an extension of Telnet.

Oh and having to redo every transfer because the default mandated by the
spec was to damage the file assuming a charset conversion was ridiculous
even for the time.

We are inventing the future here, not keeping the past alive. As usual, I
have been spending most of the two weeks between the drafts cutoff and the
IETF meeting working on my documentation tooling. Why is it that 91 people
using FTP out of over half a million is considered a serious issue? Could
we have similar concern for the people who would like to use professional
tools like Visio, Word, Powerpoint, Adobe etc to create diagrams? Right
now, there is only one tool that I know of that supports the obsolete,
deprecated, WAP-SVG required.

---

From: Larry Masinter <[email-masked]>
Date: Fri, 13 Nov 2020 09:49:29 -0800

The question isn't about how many users, as long as there are some.

It's more about the relative operational cost of setting up and maintaining
https://www.ietf.org/ietf-ftp/ rather than just keeping ftp going another
10 years.

The idea of rewriting "ftp:" URLs to be "http:" URLs doesn't help people
with scripts that were written before URLs were invented.
--
https://LarryMasinter.net @ https://Interlisp.org <= (40 year old software
with parts expecting PUP and XNS)

---

From: "John Levine" <[email-masked]>
Date: 13 Nov 2020 13:01:50 -0500

In article <[email-masked]> you write:
>> As noted in the proposal, ALL data is available at least two other ways (https://www.ietf.org/ietf-ftp  or rsync).  By request
>volume, HTTPS is massively preferred (FTP is 0.2% of HTTPS document traffic, and this is undercounting HTTPS usage) by the vast
>majority of users.  If the bulk download semantics are desired, to include incremental updates with no code required, then rsync is
>the best choice.  Do you have a user community in mind that can use FTP, but not HTTPS or rsync that we need to consider?
>

This makes no sense. What computers do you believe that people use in
2020 that support FTP but not rsync? And if that were true, why do we
see orders of magnitude more rsync traffic than FTP?

I use a Mac laptop which, based on what I've seen at IETF meetings, is
a fairly popular choice.  It has rsync as part of the base system, and
no FTP client other than as part of curl, which of course also supports
http and https.

R's,
John

---

From: Donald Eastlake <[email-masked]>
Date: Fri, 13 Nov 2020 13:09:04 -0500

Hi,

> no FTP client other than as part of curl, which of course also supports

It does have sftp :-)

Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 2386 Panoramic Circle, Apopka, FL 32703 USA
 [email-masked]

---

From: Töma Gavrichenkov <[email-masked]>
Date: Fri, 13 Nov 2020 21:11:01 +0300

Peace,

On Tue, Nov 10, 2020 at 5:24 AM Roman Danyliw <[email-masked]> wrote:
> Russ, and Alexey), Communications team (Greg),
> affected SDO liaisons, IAB Chair, and LLC ED, a
> proposed retirement plan was developed

What about the FTP sites linked to in datatracker?  E.g.
ftp://ops.ietf.org/pub/lists/idn* which is the mailing list archive
for the deceased idn-wg ( https://datatracker.ietf.org/wg/idn/about/
)?

--
Töma

---

From: "John R Levine" <[email-masked]>
Date: 13 Nov 2020 13:17:34 -0500

>> no FTP client other than as part of curl, which of course also supports

Well, sure, but as we all know, sftp and rsync both run on top of ssh.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Nov 2020 13:25:16 -0500

On 11/13/20 12:43 PM, Phillip Hallam-Baker wrote:

> On Fri, Nov 13, 2020 at 12:01 PM Keith Moore 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     FTP is a much better interface than HTTP for scripting, mirroring,
>     and remote file access (i.e. mounting an FTP server like a "share"
>     so that it can be accessed from one's computer just like any other
>     file system
>
> I disagree. The ability to mount HTTP file systems is actually built 
> into Windows. I don't think FTP is supported.

Windows?  Who uses that?  :)   People who actually want to get work done 
generally try to avoid it.

Get HTTP file systems supported by as many implementations and platforms 
as FTP use as a file system, and I'll assent to transitioning from FTP 
access to HTTP file system access.

>
> FTP is a very peculiar protocol, I have implemented it several times 
> and it was awful to do even before NATs got in the way. It is not 
> really a separate protocol, it is an extension of Telnet.

Without arguing about details, yes it's a peculiar protocol. But a 
subset that works with 99.99% of the computer systems in use today is 
not that bad, and there are several decades of accumulated understanding 
behind it.   None of which is terribly relevant to this discussion, 
since almost nobody actually has to implement FTP any more, certainly 
not the IETF.   All IETF has to do is put a server up, and plenty of 
good servers are available off-the-shelf.

> Oh and having to redo every transfer because the default mandated by 
> the spec was to damage the file assuming a charset conversion was 
> ridiculous even for the time.

Yeah, but nobody does that any more at least between reasonable 
systems.   And at least transfers between very dissimilar systems still 
work, even though they're not much needed any more.

> We are inventing the future here, not keeping the past alive.

A future of constantly assured obsolesence is not one I want to 
contribute to, and that's  exactly where the web is headed.  It's such a 
complex mess now that the number of independent effective client 
implementations is approaching 1.  If the web is the future, we're all 
doomed.    And throwing away things that work is incredibly shortsighted.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Nov 2020 13:27:33 -0500

> see orders of magnitude more rsync traffic than FTP?

1. rsync is not good for file access

2. traffic volume is not an indicator of importance.

Keith

---

From: "John Levine" <[email-masked]>
Date: 13 Nov 2020 13:37:56 -0500

In article <CALZ3u+YrxgLbibiJK9h9+na=[email-masked]> you write:
>for the deceased idn-wg ( https://datatracker.ietf.org/wg/idn/about/ )?

We should fix the links in the datatracker, since that ftp link
stopped working a long time ago. The archives for that list are
available with all of the other lists, either via the IMAP server
or the general mail and rsync archives, e.g.:

https://mailarchive.ietf.org/arch/browse/static/idn/thread/2003/

rsync://rsync.ietf.org/mailman-archive/idn

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 13 Nov 2020 13:47:14 -0500

>1. rsync is not good for file access

I suppose there is a definition of file access for which that might
still be true. I use rsync every night to mirror the rfc and i-d
archives on my laptop. Works great. It is not 1995 any more and nobody
sensible mounts a remote directory for one at a time retrieval of
small files over a WAN.

>2. traffic volume is not an indicator of importance.

At some point of course it is. We're talking about 1/5000th of the
traffic here, with strong hints that a lot of that trickle could
easily switch to other paths.

R's,
John

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Nov 2020 13:59:10 -0500

On 11/13/20 1:47 PM, John Levine wrote:

> archives on my laptop. Works great.

Sure, if you don't mind having that large a cache.   I've done that in 
the past (though with ftp as I found it worked better than rsync).  But 
even though storage is very cheap these days, it's not free (especially 
when considering the cost of backups and other maintenance).   And 
there's something to be said for not having to cache the entire RFC and 
I-D repositories just to be able to occasionally access individual files 
in a convenient way.

> It is not 1995 any more and nobody

Not for one-time retrieval, but for access anytime one wants it (and 
without having to hassle with a web browser, which already hogs too much 
screen space and attention while forcing an inflexible interface on its 
users).

And the fact that this also worked in 1995 is not a bug, but a 
feature.   Obsolescence is a bug.

> easily switch to other paths.

Emphatically disagree.   Think about it some more.   I don't know how to 
say it any more clearly.

(also, the idea that numerous parties should have to change from using 
mature, proven protocols to more complex and dubious interfaces that 
seem likely to be less stable over time, should at least be questionable)

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 13 Nov 2020 19:48:38 +0000

Hi Töma!

> From: ietf <[email-masked]> On Behalf Of John Levine
> Sent: Friday, November 13, 2020 1:38 PM
> To: [email-masked]
> In article
> <CALZ3u+YrxgLbibiJK9h9+na=[email-masked]
> > you write:
> >ftp://ops.ietf.org/pub/lists/idn* which is the mailing list archive for
> >the deceased idn-wg ( https://datatracker.ietf.org/wg/idn/about/ )?

Good catch.  In addition to this WG, we'll see about any others in the datatracker.

> We should fix the links in the datatracker, since that ftp link stopped working a
> long time ago. The archives for that list are available with all of the other lists,
> either via the IMAP server or the general mail and rsync archives, e.g.:
> 

And also https://www.ietf.org/ietf-ftp/ietf-mail-archive/idn/
(the HTTP one-to-one mapping of ftp.ietf.org)

Roman

---

From: Russ Housley <[email-masked]>
Date: Mon, 16 Nov 2020 05:43:02 -0500

I support turning off the FTP service at ietf.org. The FTP daemon places frustrating constraints on the file system structure.  HTTPS offers much greater security, and it does not impose constraints on the file system structure.

Russ

---

From: Lars Eggert <[email-masked]>
Date: Mon, 16 Nov 2020 12:51:23 +0200

On 2020-11-16, at 12:43, Russ Housley <[email-masked]> wrote:
> I support turning off the FTP service at ietf.org.

+1

Lars

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Nov 2020 09:33:12 -0500

> I support turning off the FTP service at ietf.org. The FTP daemon places frustrating constraints on the file system structure.  HTTPS offers much greater security, and it does not impose constraint…

Perhaps you could elaborate on this a bit?   It seems to me that it's 
not the FTP daemon that's placing the constraints on the file system 
structure,  but rather, the expectation that RFCs will be available at 
well-known URLs derived from the RFC number, and I-Ds will be available 
at well-known URLs derived from the I-D identifier.   Sure, a typical 
HTTP server does allow more flexibility at mapping URLs to filenames 
than a typical FTP server does, but only at the cost of a certain amount 
of operational hassle - those mappings have to be maintained.

In practice it also seems that HTTP encourages the destabilizing of such 
interfaces over time, as there's a desire to conform to the latest 
fashion in user interfaces, creating pressure to rearrange the file 
system to suit current whims.

At any rate, there's no requirement to maintain the FTP backing store 
and the HTTP backing store on the same file systems.   As John Levine 
helpfully pointed out, the amount of storage required is fairly small.   
And I understand that there's also a commitment to maintain the file 
structure and make it available using rsync, so I don't think 
deprecating the FTP service is helpful in this regard.

As for the security issue, it seems like any risk resulting from 
exposure of FTP channels to eavesdroppers is entirely on the client, and 
clients are able to choose for themselves whether this is a significant 
enough risk to migrate to a more secure, but less stable, programming 
interface.

Keith

---

From: Larry Masinter <[email-masked]>
Date: Mon, 16 Nov 2020 08:43:00 -0800

I think the cheap solution to this issue would be to build a FTP / rsync
gateway which rsync'd the available data and offered the data with FTP
services. I'd budget $500 to build such a service and $20/month to run it,
depending on how much data there is, although it's likely to cost less.  I
imagine some enterprising engineer would build it for free in exchange for
permission to include a one-line acknowledgment to their engineering
cleverness in the FTP service's greeting. Build it as a docker container
based on a small linux distro.

The only problem I see is that port forwarding from ftp://ietf.org but I
imagine someone around here could manage.
--
https://LarryMasinter.net https://interlisp.org

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Nov 2020 11:45:55 -0500

On 11/16/20 11:43 AM, Larry Masinter wrote:

> I think the cheap solution to this issue would be to build a FTP / 
> rsync gateway which rsync'd the available data and offered the data 
> with FTP services. I'd budget $500 to build such a service and 
> $20/month to run it, depending on how much data there is, although 
> it's likely to cost less.  I imagine some enterprising engineer would 
> build it for free in exchange for permission to include a one-line 
> acknowledgment to their engineering cleverness in the FTP 
> service's greeting. Build it as a docker container based on a small 
> linux distro.

Yeah, I was thinking something similar.   The biggest trick would be to 
assure integrity of the separate service, but it should be possible to 
automatically monitor it.

Keith

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 16 Nov 2020 16:47:35 +0000

There's a counter-movement among many that the IETF should not be building bespoke tools.  This seems, most notably, to come up in meetecho v other videoconf systems.

---

From: Adam Roach <[email-masked]>
Date: Mon, 16 Nov 2020 10:48:14 -0600

I do not object to turning off the FTP service.

In the analysis, I think there are two costs to consider and one 
benefit. The benefit of leaving it online, of course, is that some small 
group of users still find utility in FTP. Keith has outlined some of the 
reasons that users might prefer FTP, and I think it would be worthwhile 
replicating these properties in HTTP where possible (e.g., ensuring 
access to RFCs at a stable URL in a raw format, and possibly even 
configuring WebDAV on those paths in a read-only mode to support remote 
filesystem mounting).

The costs, as I mention, are twofold. There's a small operational cost 
to keeping the FTP service up, running, and configured. For example, if 
there were some reason to restructure the way files are stored on the 
server, it's one additional service that needs to be updated. This is 
probably pretty small from a monetary perspective.

The far greater cost is that every additional public-facing service on a 
server adds attack surface for malicious parties. And for less popular 
protocols like FTP, the chance of maintainers proactively finding and 
patching security vulnerabilities in their servers is vanishingly small. 
(If there's some vibrant community actively and continuously 
contributing to an FTP server implementation, that changes the calculus 
a fair bit, but that seems fantastically unlikely.)

So, on balance, it looks like retiring a lightly-used service is the 
right choice.

/a

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Nov 2020 11:52:21 -0500

> There's a counter-movement among many that the IETF should not be building bespoke tools.  This seems, most notably, to come up in meetecho v other videoconf systems.

Yeah, I emphatically oppose that counter-movement.    As far as I can 
tell it's basically an effort to make IETF less functional, less able to 
serve its purpose.

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Nov 2020 12:12:08 -0500

On 11/16/20 11:48 AM, Adam Roach wrote:

> benefit. The benefit of leaving it online, of course, is that some 
> small group of users still find utility in FTP.

IMO that misstates the benefit.   A stable service can have a large (and 
long-term) benefit even if only a few clients at a time use it.

Once upon a time, before RFCs were officially available in PDF, I put up 
an "RFCs in PDF" service that downloaded RFCs via FTP and automatically 
translated them to PDF.   That service basically allowed Windows users 
to print RFCs with the page breaks in the right places.   Many people 
thanked me, the RFC editor asked for my code and eventually provided 
RFCs in PDF offically.   Having a simple, stable API that I could use to 
watch for new RFCs, and download and process them, made it easy to 
implement that service.

One of the big problems with moving to HTTP is that HTTP is seen as 
implementing a user interface, and user interfaces inevitably evolve.   
The demands of providing that user interface tend to outweigh any other 
demands.

I don't have a great love of FTP except that I see tremendous value in 
an interface that has been able to be stable for decades.  To me this is 
a Good Thing, a sign that FTP got some important things right even 
though there's clearly a lot about FTP that is also baroque.   IETF 
should strive to make all of its protocols be useful for as long as FTP 
has been.

Unfortunately, HTTP shows no sign of ever getting there.   It's a lot 
more baroque than FTP ever was, and it's not stable.   It's great that 
HTTP is being improved, because it's so heavily used. But it's not 
providing a stable interface, one that works over time.

IMO, FTP should be viewed as a separate service, also a valuable one, 
rather than an alternative to HTTP.   The two have evolved differently 
to serve different purposes; neither one is suitable to replace the other.

Keith

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 16 Nov 2020 20:04:42 +0000

Hiya,

On 16/11/2020 16:52, Keith Moore wrote:
>> There's a counter-movement among many that the IETF should not be 
>> building bespoke tools.  This seems, most notably, to come up in 
>> meetecho v other videoconf systems.
> 
> Yeah, I emphatically oppose that counter-movement.Â Â Â  As far as I can 

I don't think impugning people's motives is fair there,
and asserting that they are making "an effort to make
IETF less functional" commits that sin IMO.

IIUC there are a bunch of people who think the IETF would
be better run if we used COTS products and services much
more. I don't tend to agree with that (being unfond of e.g.,
the privacy-loss and lack-of-control that I think would
ensue), but I do think those people are well-intentioned.

S.

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Nov 2020 15:14:24 -0500

On 11/16/20 3:04 PM, Stephen Farrell wrote:

>> As far as I can tell it's basically an effort to make IETF less 
>> functional, less able to serve its purpose.
>

Fair.   I don't really think that's their intent, or at least I have no 
supporting evidence of that.   But I do think that's the likely effect.

Keith

---

From: Adam Roach <[email-masked]>
Date: Mon, 16 Nov 2020 16:15:42 -0600

On 11/16/20 11:12, Keith Moore wrote:
> IMO that misstates the benefit.   A stable service can have a large 
> (and long-term) benefit even if only a few clients at a time use it.

Only read in a vacuum. If you read further, I argue for the stable 
interfaces you want. I believe that your objections (here and elsewhere) 
mentioning "user interface" conflate HTTP (transport) with HTML 
(markup), and mixing user interface into the RFCs isn't a foregone 
conclusion: the requirement you have of not mixing display in with 
documents is why I suggest that raw versions be made available. And I 
even argue for making the filesystem mountable like you requested, even 
if I consider that use case to be a bit baroque.

I'm sympathetic to the position that all change can be disruptive, and I 
understand that you know how FTP works and are comfortable with it. At 
the same time, I'm not yet seeing any requirements that you've put forth 
that can't be met pretty trivially with HTTP, other than an inferred 
bedrock requirement of "this must not change at all."

I get it. It's annoying when a technology we like falls by the wayside. 
But I think the costs I describe in my previous message are real, and 
the use cases described so far can be entirely met with HTTP. So I think 
a critical evaluation of the pros and cons points to retiring the 
service. YMMV, of course, but I encourage you to reflect a bit more on 
how much of your reaction is pushback on any change at all versus 
actually losing the ability to do what you want to do.

/a

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 16 Nov 2020 23:40:23 -0500

I use rsync to replicate rfcs and internet drafts locally.  These are 
run via crontab every night:

1 3 * * * rsync -tvz ftp.rfc-editor.org::rfcs/*.txt /home/common/ietf/rfcs
1 4 * * * rsync -tvz rsync.tools.ietf.org::tools.id/*.txt 
/home/common/ietf/drafts

Note that for drafts, I found the rsync.tools.ieft.org fqdn, but for 
rfcs I am still? using ftp.rfc-editor.org.

perhaps something like:

1 3 * * * rsync -tvz rsync.tools.ietf.org::tools.rfcs/rfc*.txt 
/home/common/ietf/rfcs

should work?

I would be happy to shift to whatever makes best sense for the 
maintainers of the repositories.

BTW, I like the local copies.  Many times on airplanes (not so common 
right now!), I needed something and there it was...

---

From: Daniel Migault <[email-masked]>
Date: Tue, 17 Nov 2020 00:03:48 -0500

Putting opex aside, it seems to me that modern communications need to be at
least authenticated so ftp cannot stay as it is. I might be missing
something, but switching ftp to https seems to me quite straight forward
and at least much easier than switching to sftp or ftps. Given the support
of https versus ftp, I also believe reducing the surface of attacks and
relying on probably better maintained code is probably a good switch. I
believe that switching to https is a good move

 Yours,
Daniel

-- 
Daniel Migault
Ericsson

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 00:25:07 -0500

On 11/17/20 12:03 AM, Daniel Migault wrote:

> Putting opex aside, it seems to me that modern communications need to 
> be at least authenticated so ftp cannot stay as it is. I might be 
> missing something, but switching ftp to https seems to me quite 
> straight forward and at least much easier than switching to sftp or 
> ftps. Given the support of https versus ftp, I also believe reducing 
> the surface of attacks and relying on probably better maintained code 
> is probably a good switch. I believe that switching to https is a good 
> move

What I'm seeing are a lot of handwaving arguments and very little 
detail, and a lot of arguments of the form "it's quite straightforward 
for all clients to rewrite their code so we don't have to run a single 
additional server".

Sure, it's possible to use webdav instead of FTP, but I am not convinced 
that results in lower opex and it definitely requires significant 
changes on the part of clients.

By "authenticated" you're only talking about authenticating servers to 
clients.   That's not useless but I'm not aware of significant efforts 
or likely efforts to substitute bogus RFC content for genuine RFC content.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 00:42:08 -0500

On 11/16/20 5:15 PM, Adam Roach wrote:

> interfaces you want.

Ok, so why not use an existing interface that has proven to be stable 
through decades of use, instead of one that _might_ be stable in the future?

> I believe that your objections (here and elsewhere) mentioning "user 
> interface" conflate HTTP (transport) with HTML (markup), and mixing 
> user interface into the RFCs isn't a foregone conclusion: the 
> requirement you have of not mixing display in with documents is why I 
> suggest that raw versions be made available. 

Sorry if I wasn't clear, but that was not my concern.  My concern is 
that the web interface taken as a whole (including not only the RFCs 
themselves but also any browsers, HTML, JS, CSS, fonts, etc. that are 
used to make up the entire user interface) are all optimized for one 
kind of user experience, and NOT for use as an API to be used by 
programs.   So if someone later decides to rearrange or otherwise alter 
the RFCs, or reconfigure servers, or whatever, in order to improve user 
experience, that will be seen as the Right Thing to do regardless of any 
concerns that people might have about the API changing.

I would argue that that's really the argument in favor of deprecating 
FTP that we are seeing now - that the stable API for accessing RFCs is 
irrelevant, because it's only the user experience via the web that 
matters.  And I do have a big problem with that.

It has been claimed that the use of FTP constrains file system layout, 
but it's mostly the API that constrains the file system layout.

> And I even argue for making the filesystem mountable like you 
> requested, even if I consider that use case to be a bit baroque.

With FTP, I can mount the RFCs as a file system this very minute (as I 
just did) and treat those RFCs as local files (say by grepping through 
them, as I just did).  With FTP it's unambiguous as to how to do this 
and the code to do this has existed for many platforms for many years.  
With WebDAV, this is also possible, but at the cost of ALL current users 
having to retool.   And I have no idea how widely WebDAV is supported - 
though I did verify that a FUSE interface to it exists for Linux.

(I don't personally consider remote file system access to be baroque at 
all; to me it seems relatively straightforward and much easier to use in 
some ways than a web service.  For instance, I can very easily download, 
or print, multiple RFCs in one command, without having to click on each 
link separately, specify its download directory and file name, etc.   
This works because FTP always was quite clearly a protocol for accessing 
files on disk. )

> I'm sympathetic to the position that all change can be disruptive, and 
> I understand that you know how FTP works and are comfortable with it. 
> At the same time, I'm not yet seeing any requirements that you've put 
> forth that can't be met pretty trivially with HTTP, other than an 
> inferred bedrock requirement of "this must not change at all."

I believe change to long-valuable services needs to be justified, and 
not made merely out of a desire to be in fashion.

I never have stated a position that all change can be disruptive.  I 
suspect that's true as a generalization, but I'm certainly not arguing 
that nothing should ever change.  I'm talking specifically about a 
single service here.

But there really should be a good reason to make such a change. I have 
run FTP servers before; they're not much trouble.  (another advantage of 
using a stable protocol).   Taken both client and server overhead into 
account, I see less overhead associated with continued use of FTP, than 
with WebDAV.   It's possible that there's even more opex on the server 
side using WebDAV, as the arguments that have been cited as detriments 
to continuing FTP might also apply to WebDAV support.

One thing that might make me change my mind might be if WebDAV were 
considerably more efficient or more functional.  That is possible.  
HTTP/webdav should require less TCP setup overhead, and does offer the 
potential for seek/read capability using byte ranges.

Another thing that might make me reconsider is if IETF wished to host 
its servers in some sort of dysfunctional cloud environment (probably 
using NAT within the cloud) that didn't support FTP's use of ports.   
But I think PASV should still work and it's pretty much the default 
these days anyway.

> I get it. It's annoying when a technology we like falls by the wayside. 

FTP hasn't fallen by the wayside.

I suspect what's happened instead is this:  There are people who think 
that the way /they/ use computers is the way /everyone/ should use 
computers.   And if so, I think they should reexamine their assumptions.

> YMMV, of course, but I encourage you to reflect a bit more on how much 
> of your reaction is pushback on any change at all versus actually 
> losing the ability to do what you want to do.

I think you're reading between the lines far more than is intended or 
warranted.

Keith

p.s. we could have set up the server in far less time than it has taken 
to write these messages.

---

From: "John Levine" <[email-masked]>
Date: 17 Nov 2020 01:24:38 -0500

In article <[email-masked]> you write:
>1 4 * * * rsync -tvz rsync.tools.ietf.org::tools.id/*.txt /home/common/ietf/drafts
>

ftp.rfc-editor.org is a CNAME for rfc-editor.org. Just delete the
"ftp." from the front of the name.

Internet Drafts are also available for rsync at
rfc-editor.org::internet-drafts although not as far back as the ones
on the tools server.

R's,
John

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 17 Nov 2020 06:40:55 +0000

Hi Bob!

> From: Robert Moskowitz <[email-masked]>
> Sent: Monday, November 16, 2020 11:40 PM
> I use rsync to replicate rfcs and internet drafts locally.  These are run via
> crontab every night:
> 
> Note that for drafts, I found the rsync.tools.ieft.org fqdn, but for rfcs I am still?
> using ftp.rfc-editor.org.
> 
> perhaps something like:
> 
> I would be happy to shift to whatever makes best sense for the maintainers of
> the repositories.

Thanks for sharing the exactly config you are using with rsync.  I see no issue with how you are syncing your files.  It's worth mentioning that there are three possible rsync servers serving different subsets of the data.  You're gets RFCs from ftp.rfc-editor.org and IDs from rsync.tools.ietf.org.  You could also get both from a single source (say rsync.ietf.org).  For those trying to find  a server closest to the ftp directory layout, the best choice is rsync.ietf.org (which also has everything-ftp, which is the entire ftp server layout).

As a PSA, this is what is found on all three servers:

$ rsync rsync.ietf.org::
everything-ftp 	- The entire IETF FTP Archive
internet-drafts	- The Internet Draft Repository (currently active drafts)
id-archive     	- The Internet Draft Archive (both active and expired drafts)
iesg-minutes   	- IESG Minutes
proceedings    	- Repository of Proceedings
xml2rfc.bibxml 	The xml2rfc citation libraries
charter        	- Repository of WG Charters
concluded-wg-ietf-mail-archive	- Older list text archives
conflict-reviews	- Repository of Conflict Review documents
iana           	- IANA assignments
iana-timezone  	- IANA Time Zone Datatbase (see also http://www.iana.org/time-zones)
legacy-files   	- Legacy material supporting long-lived URLs
mailman-archive	- Repository of Mailing List Text Archives
rfc            	- Repository of RFCs
slides         	- Repository of Slide Documents
status-changes 	- Repository of Status Change Documents

$ rsync rsync.tools.ietf.org::
tools          	The (almost) complete tools.ietf.org site.
tools.tools    	The tools collection at tools.ietf.org.
tools.html     	Currently available htmlized drafts and RFCs.
tools.html.rfc 	Currently available htmlized RFCs.
tools.pdf      	Currently available pdf conversions of drafts and RFCs.
tools.pdf.rfc  	Currently available pdf conversions of RFCs.
tools.id       	Unpurged IETF drafts repository.
tools.meeting  	Meeting materials from IETF 59 and forward
doc.history    	Document history files (html-attribute-like lines)
doc.state.attr 	Current document status in a html-attribute-like format
doc.state.822  	Current document status in a RFC822-like format
doc.state.xml  	Current document status in an xml format
doc.versions   	Document version list files
doc.meta       	All document status files
svn.group      	Group SVN repositories
xml2rfc.bibxml 	The xml2rfc citation libraries

$ rsync ftp.rfc-editor.org::
everything-ftp 	Everything FTP
refs           	XML references for RFCs (for use with xml2rfc)
rfcs           	Contents of in-notes including subdirectories std, bcp, fyi, and ien
rfcs-text-only 	Only the text files from the directories in [rfcs]
rfc-ed-all     	Entire repository (excluding internet-drafts)
internet-drafts	Internet Drafts
ids-text-only  	Only text files from the Internet Drafts mirror
rfcs-pdf-only  	PDF versions of ASCII RFCs to ensure correct page breaks, etc

Regards,
Roman

> BTW, I like the local copies.  Many times on airplanes (not so common right
> now!), I needed something and there it was...
> 
> 
>

---

From: Robert Sparks <[email-masked]>
Date: Tue, 17 Nov 2020 00:49:49 -0600

> rfc-editor.org::internet-drafts although not as far back as the ones

Please also look through the output of

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 01:20:53 -0600

On 11/16/2020 11:42 PM, Keith Moore wrote:
> But there really should be a good reason to make such a change.   I 
> have run FTP servers before; they're not much trouble.

I've run SMTP servers quite a bit. They're not much trouble. Except for 
that one time when someone hit me with some kind of Postfix buffer 
overflow exploit that allowed them to append commands to the end of 
/etc/rc.local that downloaded heaven knows what (it appeared to take 
some blunt but effective measures to hide its tracks) and which got 
bootstrapped on reboot. I lost the most part of two days rebuilding the 
server from the ground up and painstakingly restoring data from backups 
as I examined it to ensure it hadn't been corrupted by the attacker. 
(And that was presumably someone just opportunistically scanning port 25 
across the network randomly to look for vulnerable servers -- the risk 
is much higher for high-profile organizations that someone might take a 
beef with.)

The most important point that I made up-thread is that extra services 
provide extra attack surface.

That's the big cost -- and it's more significant an expense than opex by 
a fair amount. To be clear, it's a cost to be considered rather than a 
hard blocker. I mean, I still run an SMTP service on that same server, 
because it has ongoing utility for me that I can't easily replace. But I 
certainly scaled back the number of publicly-visible services I run on 
that machine after that experience. Each one added risk, and the 
amortized cost of that risk across time (as measured in potential person 
hours to recover from an attack) was generally larger than the value of 
the service for most of the services I had previously turned on.

And *that's* the calculus I'm applying in this scenario. Nothing to do 
with what's fashionable, or how I personally think everyone ought to use 
computers, or some notion that stable APIs are irrelevant. Those would 
all be questionable reasons for making a suggestion to retire a service, 
and it's uncharitable to assign such motives to me or anyone else 
without supporting evidence. I'm basing my suggestion here on a 
straight-up cost/value evaluation.

In any case, I've said my bit, and I believe my analysis holds up to the 
scrutiny you've put it through. You and others can make of my input what 
you may, but I'm really out of time to engage in a detailed 
back-and-forth here, so this is probably the last message I'll be 
sending on the topic.

/a

---

From: Sean Turner <[email-masked]>
Date: Tue, 17 Nov 2020 03:32:28 -0500

> On Nov 16, 2020, at 05:51, Lars Eggert <[email-masked]> wrote:
> 

+1

spt

---

From: Jared Mauch <[email-masked]>
Date: Tue, 17 Nov 2020 04:37:56 -0500

> On Nov 17, 2020, at 3:32 AM, Sean Turner <[email-masked]> wrote:
> 
> 

I’m reluctant to turn off services like this and still run a FTP server on my personal machine.  I do wonder if we should be also looking at moving FTP, gopher, etc. to historic?

- Jared

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 05:49:05 -0500

> provide extra attack surface.

Yeah, I get that.   I've built lots of IIoT systems that had to rely on 
a number of services, and it's always been a struggle to get the attack 
surface down to a minimum.   (The struggle is that almost nobody wants 
to spend the time/money required to do that, because security on a 
device that lives in a customer's environment is seen as Somebody Else's 
Problem.)

But, as I'm sure you're aware, moving everything to a single protocol 
doesn't necessarily address the problem so much as move it.   And 
somehow I don't think the right answer is to force people to use less 
functional protocols.   It would take more time than I've spent already 
to evaluate WebDAV's suitability to replace FTP.    There are lots of 
questions to consider, such as whether WebDAV supports symlinks, whether 
there are WebDAV clients out there that are as capable as FTP clients, 
etc.   I've never actually seen anyone use a WebDAV client.

There's also a lot that could go wrong when changing how the service is 
implemented.   Like, I've lost count of the number of patent cases I've 
participated in, in which it was extremely important that RFC dates were 
preserved.   Both FTP and HTTP support file dates, but it's harder to 
get at them using a web browser than with an FTP client.   Since the 
dates aren't as visible from HTTP, is someone going to fail to preserve 
them when copying?   I'm reminded of the statement I saw earlier that 
it's okay to regenerate the text files from the XML.   Bad Idea.

Keith

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 11:51:26 +0000

+1 (turn the ftp service off)

-----Ursprüngliche Nachricht-----
Von: ietf <[email-masked]> Im Auftrag von Sean Turner
Gesendet: Dienstag, 17. November 2020 09:32
An: ietf <[email-masked]>
Betreff: Re: Call for Community Feedback: Retiring IETF FTP Service

+1

spt

---

From: [email-masked]
Date: Tue, 17 Nov 2020 06:01:49 -0800 (PST)

The discussion of FTP service retirement has actually been surprisinginly
informative. Things I've learned include:

(1) The IETF no longer provides HTTP access, leaving FTP as the only
    access mechanism that doesn't require a crypto layer. With FTP gone,
    crypto becomes a requirement for access.

(2) Some people believe implementation of an FTP client is difficult,
    more difficult than implementing HTTPS.

(3) There is concern over the attack surface of an FTP server, but in
    practically the same breath (message) people are considering getting
    back some functionality by deploying WebDAV.

I mention (2) and (3) because I always find it interesting how people weigh
various tradeoffs.

For me the decider is (1). I don't like FTP all that much, but if it's
the only remaining access mechanism that doesn't call for crypto, I like
losing it a whole lot less.

Let's please keep the FTP servers going.

				Ned

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 17 Nov 2020 14:51:10 +0000

Hi Ned!

Thanks for the feedback.

> From: ietf <[email-masked]> On Behalf Of
> [email-masked]
> Sent: Tuesday, November 17, 2020 9:02 AM
> To: [email-masked]
> Cc: Keith Moore <[email-masked]>

Could you help me better understand which way your concern leans.  Let's abstract away HTTP and FTP, and just consider a communications channel.  Do you have a use case where access to IETF artifacts need to happen over unencrypted channels (i.e., getting the same artifacts over an encrypted channels breaks the use case)?  Put via analogy, if you always get something via postcard (in the clear), but got it in an envelope (encrypted) instead, it break something.  Or are you stating a philosophical position on not providing channel security?

Roman

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 17 Nov 2020 14:53:03 +0000

Personal views - no hats.

Time to retire the FTP service, just as other legacy protocols have been retired in the past. The IETF does not shy away from recommending that others encrypt everything, so we should take our own advice. As well, data clearly show there remains essentially no demand for FTP - users have adopted the HTTPS alternative.

More detailed rationale:
- Clearly the market has moved on. It does not cross the cost/benefit threshold to continue maintaining a service for so few connections (that all appear to be scripted machine-to-machine).
- FTP support has been removed from browser clients. As Mozilla wrote, "FTP is an insecure protocol and there are no reasons to prefer it over HTTPS for downloading resources."
- It is not encrypted. The IETF & IAB have been aggressive in pushing for pervasive encryption [1] so it is illogical that we would not make such a change on our own information resources. Per the IAB, "The IAB now believes it is important for protocol designers, developers, and operators to make encryption the norm for Internet traffic."

[1] https://mailarchive.ietf.org/arch/msg/ietf-announce/ObCNmWcsFPNTIdMX5fmbuJoKFR8/

- Jason

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 08:55:09 -0600

Yep. WebDAV is generally a configuration option on an existing HTTP server rather than a whole new server. There’s a chance that WebDAV-specific codepaths could enable some attacks that wouldn’t otherwise be possible, but the fact that the vast majority of its code is shared with the rest of the HTTP server’s functionality makes the chance of that happening minuscule compared to running a whole separate codebase.

/a

> On Nov 17, 2020, at 08:16, [email-masked] wrote:
>

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 17 Nov 2020 14:57:52 +0000

One reply inline below

On 11/17/20, 9:16 AM, "ietf on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

>  I don't like FTP all that much, but if it's the only remaining access mechanism that doesn't call for crypto, I like losing it a whole lot less.

[JL] But our position on new protocols & deployers of IETF standards is that everything needs to be encrypted. So in light of this, why would we then except our own activities from this shift to pervasive encryption? As well, I do not understand the burden on the end user in terms of access with crypto - every browser supports it, every mobile device, etc. The decision to retire FTP seems like the easiest decision the IETF can make this year.

Jason (personal views - no hats)

---

From: [email-masked]
Date: Tue, 17 Nov 2020 06:53:19 -0800 (PST)

> Could you help me better understand which way your concern leans.  Let's
> abstract away HTTP and FTP, and just consider a communications channel.  Do you
> have a use case where access to IETF artifacts need to happen over unencrypted
> channels (i.e., getting the same artifacts over an encrypted channels breaks
> the use case)? 

For myself, I routinely use devices that are incapable of supporting a crypto
stack that would work with an IETF HTTPS server. I haven't had the need to
access IETF resources recently from such hardware, which is why I hadn't
noticed this change. But that's happenstance, nothing more.

However, the case that worries me much more is how this may affect access from
places where crypto use is problematic.

> Put via analogy, if you always get something via postcard (in the clear), but
> got it in an envelope (encrypted) instead, it break something.  Or are you
> stating a philosophical position on not providing channel security?

I'm well aware of Zimmerman's postcard/letter analogy. I'm also aware of the
responses to that analogy that pointed out that things are nowhere near
as simple as the analogy would have you believe.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 10:45:13 -0500

> Time to retire the FTP service, just as other legacy protocols have been retired in the past. The IETF does not shy away from recommending that others encrypt everything, so we should take our own…
I cannot say this often enough:   Traffic volume is not an indicator of 
importance.
> - Clearly the market has moved on. It does not cross the cost/benefit threshold to continue maintaining a service for so few connections (that all appear to be scripted machine-to-machine).
There is not one "market".   FTP is a different service than the web, 
with distinct advantages over the web.   And traffic volume is not an 
indicator of importance.
> - FTP support has been removed from browser clients. As Mozilla wrote, "FTP is an insecure protocol and there are no reasons to prefer it over HTTPS for downloading resources."
I personally find that unfortunate, but support in browser clients is 
not an indicator of FTP's utility either.   One reason to use FTP is 
that browser clients are really poor tools for some kinds of file 
transfer, especially if you want to transfer multiple files with minimum 
human interaction.
> - It is not encrypted. The IETF & IAB have been aggressive in pushing for pervasive encryption [1] so it is illogical that we would not make such a change on our own information resources. Per the…

"the norm" != "required".   I'd be happy to see a version of FTP that 
supports encrypted transmission as an option, as long as it were 
optional.   (Are those web browsers that are deprecating FTP also 
deprecating HTTP without TLS?)

And as Ned pointed out, there are still reasons to use unencrypted 
transmission on occasion.

Also, perhaps the IETF and IAB should be a bit less dogmatic, in light 
of experience.  I keep seeing situations in which deprecation of old TLS 
versions is breaking systems for which there is no browser that supports 
the new TLS versions.  IMO this does significant harm.   I realize some 
people believe in planned obsolescence, but I don't think they have a 
good case.

Keith

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 09:57:34 -0600

On 11/17/20 09:45, Keith Moore wrote:
> Are those web browsers that are deprecating FTP also deprecating HTTP 
> without TLS?

Yes.

https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-http/

https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure

/a

---

From: Jared Mauch <[email-masked]>
Date: Tue, 17 Nov 2020 11:00:02 -0500

> > Personal views - no hats.

	Similar.

> There is not one "market".   FTP is a different service than the web, with
> distinct advantages over the web. 

	The transport area begs to differ.

	It's clear that UDP is the one-true-way for all future transports.
I continue to hold what appears to be a minority view which is that
UDP/QUIC is harmful.

	It's clear many parts of the IETF have become the web transport
engineering task force.  If you don't subscribe to this, you will
find yourself ignored.  

	If the IETF feels the protocols are not viable, lets properly move
them to historic.  I don't expect someone to stand up a gopher server
in 2021 any more than I expect someone to set up a FTP server in 2021.

	- Jared

---

From: Jared Mauch <[email-masked]>
Date: Tue, 17 Nov 2020 11:01:55 -0500

> https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure

	There's a difference between preferring https vs http and pulling
http support entirely.  There's many devices that will never get https,
upgrades or certificates.

	We must be careful in how quickly we drag viable technology
to the digital (and perhaps physical) dumpster.

	- Jared

-- 
Jared Mauch  | pgp key available via finger from [email-masked]
clue++;      | http://puck.nether.net/~jared/  My statements are only mine.

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:02:11 -0500

> https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure 

Wow.  That's incredibly arrogant and shortsighted.  I cannot begin to 
count, for instance, the number of Internet appliances out there (in 
both consumer and industrial applications) that have http interfaces but 
do not support https.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:03:29 -0500

> in 2021 any more than I expect someone to set up a FTP server in 2021.

FTP is still quite useful, there are many clients out there still in 
use.  gopher, not so much.

Keith

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Tue, 17 Nov 2020 11:03:29 -0500

On Tue, Nov 17, 2020 at 12:25:07AM -0500, Keith Moore wrote:
> What I'm seeing are a lot of handwaving arguments and very little detail,
> and a lot of arguments of the form "it's quite straightforward for all
> clients to rewrite their code so we don't have to run a single additional
> server".

So I don't really care *all* that much, but to be fair, there have
been some hand-waiving arguments on both sides.

The most common way that *I* would write scripts to download files,
going back at least ten years, using ftp is to use either curl or wget
--- and both of those scriptable, command-line tools support both http
and ftp.  So transitioning such scripts to use http versus ftp is not
hard, and given the problems with NAT boxes, I would have transitioned
over my scripts using wget or curl to use http a long time ago.

And while it is true that the http *protocol* has been evolving, it is
*not* a "user interface"; it is a protocol which supports user
interface programs, and as a protocol there has been quite good
backwards compatibility, since people (especially in government) have
insisted on running Windows 95 long past when it was safe and sane to
do so (speaking as someone whose SF-86 was compromised as a result),
and so web servers will support ancient http clients without any
issues that I'm been made aware of.

So the argument that http is "unstable" and so we should stick with
supporting a file transfer protocol invented in the 197o's because it
is somehow more "stable" is also, in my opinion, full of hand-waving.

Cheers,

						- Ted

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:10:16 -0500

On 11/17/20 11:03 AM, Theodore Y. Ts'o wrote:

> And while it is true that the http*protocol*  has been evolving, it is
> interface programs,

To clarify, my point is that the provisioning and configuration of 
HTTP[S] services is driven by user interface considerations, since 
that's how people view "the web". (even though "the web" intends and 
always was intended to support more than just user interfaces).   That's 
why, for instance, web content often breaks old URLs and the content 
authors/providers feel justified in doing so.

It's hard to make the same protocol suite serve multiple masters.

> and as a protocol there has been quite good

What I've seen is that for http (not https) the backward compatibility 
in browsers has to date been very good, but the deprecation of older 
versions of TLS has broken lots of things. Anyway, that's what I was 
thinking of when I said "unstable". And now I learn that browsers are 
dropping support for http, which I think also supports the argument that 
the web protocol suite is unstable.

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 17 Nov 2020 16:10:15 +0000

Hi Jared!

> From: ietf <[email-masked]> On Behalf Of Jared Mauch
> Sent: Tuesday, November 17, 2020 11:02 AM
> To: Adam Roach <[email-masked]>
> Cc: Keith Moore <[email-masked]>; [email-masked]
> > > Are those web browsers that are deprecating FTP also deprecating
> > > HTTP without TLS?
> >
> >
> > Yes.
> >
> > https://www.chromium.org/Home/chromium-security/marking-http-as-non-
> se
> > cure
> 
> 	There's a difference between preferring https vs http and pulling http
> support entirely.  There's many devices that will never get https, upgrades or
> certificates.

I can see this point in the abstract.  Can you help me in the specific -- what is the expected configuration of device accessing IETF resources which is not HTTPS capable?

Roman

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:24:32 -0500

> I can see this point in the abstract.  Can you help me in the specific -- what is the expected configuration of device accessing IETF resources which is not HTTPS capable?

I agree that there are many devices that have http servers that will 
never get https etc.   So it's very unfortunate that browser vendors are 
planning to deprecate http support, and I predict this will be extremely 
disruptive and harmful if it's carried out.

But I don't think that specific concern is relevant to the question 
about whether IETF should deprecate FTP or even HTTP [no s] services for 
IETF.   The appliances that have built-in http servers won't affect 
IETF's servers.

Keith

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 17 Nov 2020 16:25:14 +0000

[JL] It sounds like those appliances have not had any software updates for many years - perhaps a decade or more. Perhaps they are already compromised or soon will be.

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:34:35 -0500

> [JL] It sounds like those appliances have not had any software updates for many years - perhaps a decade or more. Perhaps they are already compromised or soon will be.

Wow.

There are many environments for which software updates are 
infeasible.     Many devices are deliberately operated on airgapped 
networks, which doesn't mean they're entirely immune to attack but it 
does mean that relying on updates from the net doesn't work.   It's 
extremely expensive to support a low-volume device with security 
updates.   Updates are quite reasonably seen as threats to operations in 
manufacturing environments, where any disruption costs real money, and 
any updates to firmware would require comprehensive retesting of the 
entire system.   And when a device doesn't need frequent updates, any 
update is likely to be done by people who aren't familiar with the code, 
and the chance of introducing more bugs than are fixed is quite high.

So basically, it's completely unrealistic to assume that every device 
should be updated, or that updates improve the security of a device.

Keith

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 17 Nov 2020 16:42:04 +0000

On 11/17/20, 10:46 AM, "ietf on behalf of Keith Moore" <[email-masked] on behalf of [email-masked]> wrote:
> I cannot say this often enough:   Traffic volume is not an indicator of  importance.

[JL] Fair enough. But we also have a count of the number of hosts using the FTP interface and that is a quite small number. Most seem to be scripted, presumably to create a mirror of the RFCs.

    of experience.  I keep seeing situations in which deprecation of old TLS
    versions is breaking systems for which there is no browser that supports
    the new TLS versions.  IMO this does significant harm.

[JL] The IETF regularly standardizes new protocols that leads to "creative disruption" globally. This brings countless positive benefits, such as pervasive encryption or real-time voice and video communications, but it surely has some downsides/impacts. It seems odd that we may be unable to do just a little bit of what we ask others to do in terms of disruption and proceed forward by dropping legacy unencrypted protocols with no apparent user demand & embracing more modern and secure communication protocols. BTW FTP was first specified 49 years ago in 1971. ;-)

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 11:53:49 -0500

> [JL] Fair enough. But we also have a count of the number of hosts using the FTP interface and that is a quite small number. Most seem to be scripted, presumably to create a mirror of the RFCs.

Ok, but don't equate "host" to "some user's PC".   A "host" could be 
operating a service that is valuable to many.   And in general, it seems 
like it's in IETF's interest to facilitate development of services that 
use RFCs.

Though I suppose that new services could reasonably use WebDAV if 
reasonable libraries and scriptable programs exist to access WebDAV 
servers.    But the original proposal was to deprecate file-system like 
access entirely, not to substitute WebDAV for FTP.

> [JL] The IETF regularly standardizes new protocols that leads to "creative disruption" globally. This brings countless positive benefits, such as pervasive encryption or real-time voice and video communications, but it surely has some downsides/impacts. It seems odd that we may be unable to do just a little bit of what we ask others to do in terms of disruption and proceed forward by dropping legacy unencrypted protocols with no apparent user demand & embracing more modern and secure communication protocols.
Maybe we shouldn't be so eager to break things that work.   Maybe, 
especially, we shouldn't assume that we're in a position to know the 
right answer to every user's needs and every use case.

> BTW FTP was first specified 49 years ago in 1971. ;-)

Yeah, isn't it great that FTP has worked for so long?    New is not 
generally better, and often it's worse.   HTTP has always lacked 
functionality that had been present in FTP for decades. That was mostly 
fine as long as FTP was still supported.   I have used FTP several times 
a year for several years (often in patent work) because in those 
instances it worked better than HTTP and a browser.

Keith

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 17 Nov 2020 09:57:38 -0700

> both consumer and industrial applications) that have http interfaces but

Keith, you said something similar on the UTA WG list earlier this year
when we talked about adding a work item to revise BCP 195 in the light
of TLS 1.3. It would be helpful if you could explain your thinking in
more detail. Are you concerned that web browsers which eventually
deprecate HTTP without TLS will make it impossible for people to
interact with certain deployed Internet appliances? Do note that when
the time comes such web browsers will provide an escape hatch: they
won't make it impossible to use HTTP without TLS, but they will force
the user to make an explicit decision about setting up an unencrypted
connection. Here again (as with Adam Roach's messages about the IETF's
FTP service) it's a question of tradeoffs and cost/benefit analysis.
Because the vast majority of web browsing activity involves interacting
with sites on the open web, not with Internet appliances, it seems
reasonable to protect users during such interactions to prevent a wide
array of attacks and abuses, from password sniffing to eavesdropping to
tracking and profiling. However, also giving users the ability to
explicitly choose unencrypted connections in certain special
circumstances seems to me to strike the right balance.

Peter

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 12:10:09 -0500

On 11/17/20 11:57 AM, Peter Saint-Andre wrote:

> more detail.

Yeah, I've been working on a draft about the realities of (mostly-) 
disconnected LANs.  I need to get it out the door.

> Are you concerned that web browsers which eventually
> connection.

I'm all for making such decisions more visible to users, but as we all 
know popup dialogs tend to get misunderstood - some people just blindly 
click-through, some people think something's wrong when there's really 
nothing wrong.   Not sure how to solve that problem.

> Here again (as with Adam Roach's messages about the IETF's
I don't think we've really seen any such analysis in regard to the FTP 
service, partly because it's hard to know just what those FTP clients 
are doing with the RFCs they download.
> circumstances seems to me to strike the right balance.

The devil is in the details, especially in user interface details.   But 
I'm not encouraged by the behavior of old browsers that can't talk to 
newer versions of TLS.   In my experience, they simply refuse to work at 
all.   They don't give the user any options.

Keith

---

From: Julian Reschke <[email-masked]>
Date: Tue, 17 Nov 2020 18:21:25 +0100

Am 17.11.2020 um 17:10 schrieb Keith Moore:
> ...
> ...

They are? Details please.

 > ...

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 17 Nov 2020 18:24:55 +0100

> both consumer and industrial applications) that have http interfaces but

Marking something as "insecure" is not the same thing as dropping support.

Best regards, Julian

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 17 Nov 2020 17:40:04 +0000

> From: ietf <[email-masked]> On Behalf Of Livingood, Jason
> Sent: 17 November 2020 16:42
> To: Keith Moore <[email-masked]>; [email-masked]
> On 11/17/20, 10:46 AM, "ietf on behalf of Keith Moore" <ietf-
> [email-masked] on behalf of [email-masked]> wrote:
> [JL] Fair enough. But we also have a count of the number of hosts using
> the FTP interface and that is a quite small number. Most seem to be
> scripted, presumably to create a mirror of the RFCs.
> 
> [JL] The IETF regularly standardizes new protocols that leads to "creative
> disruption" globally. This brings countless positive benefits, such as
> pervasive encryption or real-time voice and video communications, but it
> surely has some downsides/impacts. It seems odd that we may be unable to
> do just a little bit of what we ask others to do in terms of disruption
> and proceed forward by dropping legacy unencrypted protocols with no
> apparent user demand & embracing more modern and secure communication
> protocols. BTW FTP was first specified 49 years ago in 1971. ;-)
> 
[RW] 

If the world ever reaches the stage where only 0.2% of traffic is IPv4 and rest is IPv6 (or newer) then I wonder if ISPs will be willing to keep the IPv4 service up and running?

Clearly there is some cost to keeping the FTP service running, but:
  - Roman's analysis seems to indicate that it is used by very few individuals,
  - there are viable alternatives (e.g., rsync, curl, wget, https), and
  - Roman has even kindly offered help for folks to migrate, if required.

I think that it is great that IETF has managed to offer the FTP service for so long, but on balance, it seems to have reached a point where the benefits no longer outweigh the costs, and pragmatically it seems like it is time to turn it off.

Regards,
Rob

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 14:43:25 -0500

Ned,

>      crypto becomes a requirement for access.

I do not use crypto with rsync.

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 14:46:01 -0500

> Could you help me better understand which way your concern leans.  Let's abstract away HTTP and FTP, and just consider a communications channel.  Do you have a use case where access to IETF artifac…

Crypto channels are all well and good.  I just do not see the arguement 
to REQUIRE them for open documents like IETF rfcs and drafts.  The 
OPTION if someone is concerned that their retrieval needs protection, 
OK, but a MUST?

I just don't see it.

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 14:50:44 -0500

> [JL] But our position on new protocols & deployers of IETF standards is that everything needs to be encrypted. So in light of this, why would we then except our own activities from this shift to pe…

RSYNC is a great tool for maintaining a local document store. Running 
RSYNC over TLS is a bit of a challenge.  Will you provide this service?

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 14:53:01 -0500

On 11/17/20 10:45 AM, Keith Moore wrote:
>> Time to retire the FTP service, just as other legacy protocols have 
>> been retired in the past. The IETF does not shy away from 
>> recommending that others encrypt everything, so we should take our 
>> own advice. As well, data clearly show there remains essentially no 
>> demand for FTP - users have adopted the HTTPS alternative.
>> - Clearly the market has moved on. It does not cross the cost/benefit 
>> threshold to continue maintaining a service for so few connections 
>> (that all appear to be scripted machine-to-machine).
>> - FTP support has been removed from browser clients. As Mozilla 
>> wrote, "FTP is an insecure protocol and there are no reasons to 
>> prefer it over HTTPS for downloading resources."
> transfer, especially if you want to transfer multiple files with 
> minimum human interaction.

I find wget a great tool as well.  I have used it on sites with multiple 
documents that do not support other mechanisms.  I have scripts running 
wget for some of these.  I just see rsnyc as a more efficient use of 
resources than a script running wget(s).

>> - It is not encrypted. The IETF & IAB have been aggressive in pushing 
>> for pervasive encryption [1] so it is illogical that we would not 
>> make such a change on our own information resources. Per the IAB, 
>> "The IAB now believes it is important for protocol designers, 
>> developers, and operators to make encryption the norm for Internet 
>> traffic."
>
> of experience.  I keep seeing situations in which deprecation of old 
> TLS versions is breaking systems for which there is no browser that 
> supports the new TLS versions.  IMO this does significant harm.   I 
> realize some people believe in planned obsolescence, but I don't think 
> they have a good case.
>
> Keith
>
>

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 14:53:33 -0500

> If the world ever reaches the stage where only 0.2% of traffic is IPv4 and rest is IPv6 (or newer) then I wonder if ISPs will be willing to keep the IPv4 service up and running?

I'll answer the question, but first, explain why I don't think it's a 
valid analogy:   HTTP and FTP are not equivalent services; there are 
useful features in FTP for which equivalents do not exist in vanilla 
HTTP.   IPv6 is designed as a near-equivalent to IPv4; the only useful 
feature it really sacrifices is inability to directly talk to IPv4 
hosts.  Also, IETF is not a business and for most purposes should not 
try to act like one.

(Now as for the ISP question:  I presume ISPs will continue to support 
v4 as long as there's sufficient paying market.  And when there's no 
longer sufficient paying market to buy IPv4 directly from one's ISP, 
there will presumably be boxes for sale that can manage tunnels from 
one's enterprise LAN to private v4 networks and/or to whatever's left of 
the public IPv4 network.   It doesn't take much of a market to support 
the creation and maintenance of such boxes.   I suspect we'll see 
wide-area IPv4 networking of one kind or another for another 20 years; 
perhaps even longer than within enterprise networks. )

>    - Roman's analysis seems to indicate that it is used by very few individuals,
very few hosts != very few individuals
>    - there are viable alternatives (e.g., rsync, curl, wget, https), and

none of those are functionally equivalent to the FTP protocol.

The proposal to support WebDAV instead is somewhat more interesting, but 
details would need to be worked out.

Keith

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 14:54:18 -0500

> https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure

This causes grief with firewall authentication where plain http is 
needed for the firewall to intercept and force authentication.

How to shoot ourselves in the foot.

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 14:59:09 -0500

On 11/17/20 2:53 PM, Robert Moskowitz wrote:

> I find wget a great tool as well.  I have used it on sites with 
> multiple documents that do not support other mechanisms.  I have 
> scripts running wget for some of these.  I just see rsnyc as a more 
> efficient use of resources than a script running wget(s).

I use wget on occasion.  But it doesn't let me browse a file system, 
interactively list files, etc.   Nor does it let me mount an FTP server 
as a file system.   wget works well in some kinds of scripts, as does 
curl, but it's not exactly easy to use.

(I'm pretty sure you know this.   But some people seem to be saying "you 
don't need ftp, you can use this other tool for scripting" when the 
underlying protocol doesn't support the features that the particular 
application needs, and the applications using FTP aren't always scripts 
calling a separate program.)

Keith

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 14:11:38 -0600

> needed for the firewall to intercept and force authentication. 

See also https://datatracker.ietf.org/wg/capport/charter/

/a

---

From: [email-masked]
Date: Tue, 17 Nov 2020 12:04:07 -0800 (PST)

> needed for the firewall to intercept and force authentication.

The present situation with HSTS latching causes grief. The internal web
I used to use that was reliably http-only recently upgraded, forcing me
to find a different one.

HTTP support disappearing entirely, OTOH, is s disaster.

> How to shoot ourselves in the foot.

That's not your foot.

				Ned

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 17 Nov 2020 15:30:38 -0500

> I'm very opposed to this proposal.

Likewise.

> hideously complex.???? Use of FTP as a remote file access protocol is very
> widely supported.

It is, and part of the reason for that is the stability you reference.
Scripts that work tend to keep working for a very long time.

Another reason for keeping FTP around (that I haven't seen articulated
in this thread; please forgive me if I missed it) is that having a second
or third way to access information is extremely useful when things go
very wrong.  If that happens even once over the next 20 years then the
tiny cost of keeping FTP running will be repaid in full very quickly.

---rsk

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 15:44:38 -0500

> I can see this point in the abstract.  Can you help me in the specific -- what is the expected configuration of device accessing IETF resources which is not HTTPS capable?

Old CPE firewalls that proxy the user request and cannot support http?  
But with other services all moving to https, I suspect they are rapidly 
being upgraded.

Initial proxy authentication is a problem, though.

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 15:51:46 -0500

> circumstances seems to me to strike the right balance.

Changing the behavior of knowing when you have to use encrypt to knowing 
when you can't encrypt is good policy moving forward.  The user 
interaction needs to be simple in such cases.

Not allowing no encryption is bad business.

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 15:58:59 -0500

> See also https://datatracker.ietf.org/wg/capport/charter/

And now you just have to upgrade all those CPEs...

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 17 Nov 2020 21:08:49 +0000

I'm generally ok with us turning off FTP and don't onsider
it that big a deal either way.

However, I have to say most of the arguments for and against
doing that I've seen in this thread are weak to very weak
and some I'd consider just bogus or exaggerated. That said,
this cost/benefit argument for maintaining the service seems
worth considering:

On 17/11/2020 20:30, Rich Kulawiec wrote:
> is that having a second

I guess a lot would have to go wrong for a sustained period
for FTP to save the day, but I could just about imagine it
happening. It's not quite movie-plot time but fracturing in
the root stores causing HTTPS to not work everywhere, plus
some SSH bugginess that affected most clients and broke
rsync/SSH might do it. And as we've seen this year, now
and then stuff does hit the fan.

I don't know if it's a sufficiently strong argument but it
is one I'd not considered.

Cheers,
S.

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 17 Nov 2020 16:20:00 -0500

> Scripts that work tend to keep working for a very long time.

By now I think I also qualify as old timer, and yet I find the claim
that FTP is simple while HTTP is "complex" far from credible.  If
anything, the reverse is actually true.  With binary vs. text modes,
passive vs.  active modes, separate control and data channels, FTP is
far from "simple".  And HTML is not relevant to the discussion.  The
content type is same (either plain text or HTML) regardless of the
transfer method.

> tiny cost of keeping FTP running will be repaid in full very quickly.

This too I find not very credible.  The IETF is not making money serving
files to high-paying customers who'll bolt to a competitor if some tight
SLA is not met.  It is difficult to imagine content hosted by ietf.org
that is so critical, and also not available from multiple sources, that
we can't wait out a brief outage.

HTTP is now ubiquitous, and retiring FTP can I think be reasonably
justified given the reported light and declining usage.

-- 
    Viktor.

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 15:32:14 -0600

> RSYNC over TLS is a bit of a challenge.  Will you provide this service?

It's not an exact analog, but if  you want an authenticated channel, the 
following provides the same basic functionality:

wget -nv -r -N -l0 -np -nH --cut-dirs=1 -A txt,pdf 
https://www.ietf.org/ietf-ftp/rfc/

It could be tuned up a bit by asking the tools team to turn 
"Last-Modified"/"If-Modified-Since" header handling on for 
https://www.ietf.org/ (which would make the command above act 
substantially more like rsync).

/a

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 16:51:50 -0500

> substantially more like rsync).

Last-Modified (at least) should be regarded as absolutely essential.  
Again, these dates are sometimes important for patent litigation purposes.

(It would be worth keeping FTP for that reason alone if the HTTP server 
doesn't support Last-Modified)

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 16:59:37 -0500

On 11/17/20 4:20 PM, Viktor Dukhovni wrote:

> far from "simple".

True, but these days you mostly just use PASV and TYPE I (unless perhaps 
the client detects some baroque OS).   Clever clients try to handle any 
kind of FTP server that might exist, but a client doesn't need to 
support the whole protocol if it knows it's talking to a UNIX/Linux 
system.  (I'm not particularly worried about IETF migrating its servers 
to MCP...)

For comparison, HTTP redirect handling can be somewhat hairy, and add 
TLS on top of that which is quite often necessary these days.   Probably 
most simple clients can get away with using HTTP/1.1 today but who knows 
how long that will last.

> And HTML is not relevant to the discussion.  The

The relevance is only if you need to parse the HTML, say, to figure out 
the names of files on the server.   Which you often do need to do.   But 
that illustrates a problem with HTTP (sans WebDAV) - the content 
intended for humans isn't clearly distinct from the information needed 
by the client.  (Say you're trying to download new RFCs - the client 
needs to know which <A>'s refer to RFCs and which refer to, say, 
navigation around the site.   It's not rocket science but it is 
annoying, especially given that web sites change from time to time.)

Keith

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 17:20:22 -0500

On 11/17/20 4:32 PM, Adam Roach wrote:
> It's not an exact analog, but if  you want an authenticated channel, 
> the following provides the same basic functionality:
>

This is very similar to what I use when I am physically at an IEEE 802 
meeting and Rick has a server I can hit with wget.

The command works similiarly but is not efficient.  And for the shear 
size of the ID directory not a wise approach over rsync.

---

From: Adam Roach <[email-masked]>
Date: Tue, 17 Nov 2020 16:23:18 -0600

> Again, these dates are sometimes important for patent litigation 

The secretariat will provide authenticated versions of drafts and RFCs 
that include authenticated dates. For any ligation, I would definitely 
use these in preference to any server-provided dates.

/a

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 17:26:30 -0500

> use these in preference to any server-provided dates. 

IANAL, but I've had L clients who expressed a preference for server 
dates.   Anyway, I still maintain it's important to preserve dates, just 
so that there's less reason for suspicion that the documents themselves 
have been altered.

Keith

---

From: Robert Moskowitz <[email-masked]>
Date: Tue, 17 Nov 2020 17:21:31 -0500

On 11/17/20 4:51 PM, Keith Moore wrote:
> (It would be worth keeping FTP for that reason alone if the HTTP 
> server doesn't support Last-Modified)

Another reason why I like rsync in that I can easily maintain the server 
date on my local store.

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 17 Nov 2020 15:46:13 -0700

> so that there's less reason for suspicion that the documents themselves

I suggest that we leave such requirements to counsel.

Peter

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Tue, 17 Nov 2020 17:47:52 -0500

On Tue, Nov 17, 2020 at 04:51:50PM -0500, Keith Moore wrote:
> Last-Modified (at least) should be regarded as absolutely essential.  Again,
> these dates are sometimes important for patent litigation purposes.
> 

Whether those dates are returned by http or ftp, they're going to be
trivially foreable, and I suspect that for the purposes of patent
litigation, things like the Internet Archive and other mechanisms such
as personal e-mail archives plus the sworn testimony from owner of
said e-mail archive.

					- Ted

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 17:58:34 -0500

On 11/17/20 5:47 PM, Theodore Y. Ts'o wrote:

> trivially foreable

So are the dates in old email and usenet messages.  And yet, such dates 
are routinely cited.

(I don't claim to understand a court's or patent office's reasoning 
about what makes a date valid or not valid - their logic is different 
than those of engineers. )

I was also once involved in a case where the RFC Editor screwed up and 
the mod date on an RFC on their server was incorrect.   The declaration 
was edited to omit mention of that RFC to support the argument, even 
though the RFC editor attested as to the correct date, fixed the mod 
date on the server, and verified that their copy exactly matched the 
original.

So basically - preserve the dates and make sure they're available via 
the protocol.   It's not difficult.

Keith

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Tue, 17 Nov 2020 18:11:00 -0500

On Tue, Nov 17, 2020 at 05:58:34PM -0500, Keith Moore wrote:
> So are the dates in old email and usenet messages.  And yet, such dates are
> routinely cited.

Note that I said that this would be combined with the **sworn**
testimony of the owner of said e-mail archive.

> (I don't claim to understand a court's or patent office's reasoning about
> what makes a date valid or not valid - their logic is different than those
> of engineers. )

Yes, which is why it's not particularly a good argument for preserving
FTP on IETF's servers.

As others have said, most of the arguments on this are pretty weak on
both the "pro" and "anti" side.

					- Ted

---

From: Michael Thomas <[email-masked]>
Date: Tue, 17 Nov 2020 15:18:26 -0800

> both the "pro" and "anti" side.
All of the best mile long treads are.

Mike

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 18:22:01 -0500

> testimony of the owner of said e-mail archive.
I assume you've also done some of this before.    As I understand it 
there are practical limits on the number of witnesses that can be 
called, some witnesses cannot be used because they've assisted other 
parties in the case, etc.   All I know is that I've had law firm clients 
ask for such dates and be willing to cite them.
> both the "pro" and "anti" side.

Well, I do have experience to draw from about this.   So I disagree.

Keith

---

From: "John Levine" <[email-masked]>
Date: 17 Nov 2020 18:38:57 -0500

>Again, these dates are sometimes important for patent litigation purposes.

We have a process to respond to requests for document authentication
and I can assure you it does not involve FTP at all, nor does it
depend on any dates in HTTP headers.

You can learn all about it here:

https://ietf.org/about/administration/legal-request-procedures/

R's,
John

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Nov 2020 18:41:47 -0500

> We have a process to respond to requests for document authentication

as the subject implies, that's not a reason to trash mod dates on files.

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 18 Nov 2020 11:00:09 +0000

Hi Ned!

> From: Ned Freed <[email-masked]>
> Sent: Tuesday, November 17, 2020 9:53 AM
> Cc: [email-masked]; Keith Moore <moore@network-
> heretics.com>; [email-masked]
> > > The discussion of FTP service retirement has actually been
> > > surprisinginly informative. Things I've learned include:
> > >
> > Could you help me better understand which way your concern leans.
> > Let's abstract away HTTP and FTP, and just consider a communications
> > channel.  Do you have a use case where access to IETF artifacts need
> > to happen over unencrypted channels (i.e., getting the same artifacts
> > over an encrypted channels breaks the use case)?
> 

Thanks for clarifying.  This access must not have had happened in several years.  HTTP has been turned off since 2015  per https://www.ietf.org/about/groups/iesg/statements/maximizing-encrypted-access/.

> However, the case that worries me much more is how this may affect access
> from places where crypto use is problematic.

We haven't been serving over HTTP in 5 years.  As I understand it, the concern is that FTP is one of the last ways to access IETF information unencrypted.  Do I have it wrong?  As I responded to Toerless [1], the primary users of FTP (by volume) don't appear to be disadvantaged:

==[ snip from [1] ]==
==[ snip ]==

Hence, from what we see from actual usage, those disadvantaged users appear largely hypothetical.

Furthermore, >99% of requests are syncing the entire or parts of the repository which can be done with rsync.  As pointed out by Bob [2], rsync can be used over rsh (unencrypted):

==[ snip ]==
$ rsync rsh rsync.ietf.org::
==[ snip ]==

Roman

[1] https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5dAb3FAgNng/
[2] https://mailarchive.ietf.org/arch/msg/ietf/maUHi4gfaPAH_TfsU6XbqdYYM5Y/

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 18 Nov 2020 11:10:27 +0000

Hi Bob!

> Sent: Tuesday, November 17, 2020 3:45 PM
> To: Roman Danyliw <[email-masked]>; Jared Mauch <[email-masked]>
> >>> https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-
> >>> ht
> >>> tp/
> >>>
> >> http support entirely.  There's many devices that will never get
> >> https, upgrades or certificates.
> > I can see this point in the abstract.  Can you help me in the specific -- what is
> the expected configuration of device accessing IETF resources which is not
> HTTPS capable?
> 
> Old CPE firewalls that proxy the user request and cannot support http? But
> with other services all moving to https, I suspect they are rapidly being
> upgraded.

So to clarify, the thinking is that these devices then are falling back to FTP since they haven't been able to use HTTP in 5 years?  As noted in [1] and [2], the data doesn't seem to support this as FTP usage appears to be coming from fairly robust mature operations (based on IPs).

Regards,
Roman

[1] https://mailarchive.ietf.org/arch/msg/ietf/b8BfvrcpLmvvjkhJ1MW8DUEzmQ8/
[2] https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5dAb3FAgNng/

Roman

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 18 Nov 2020 06:17:19 -0500

On 11/18/20 6:00 AM, Roman Danyliw wrote:
> We haven't been serving over HTTP in 5 years.  As I understand it, the concern is that FTP is one of the last ways to access IETF information unencrypted.  Do I have it wrong?

rsync is unencrypted.

>    As I responded to Toerless [1], the primary users of FTP (by volume) don't appear to be disadvantaged:
>
> ==[ snip from [1] ]==

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 07:05:35 -0500

> As I responded to Toerless [1], the primary users of FTP (by volume) don't appear to be disadvantaged:

The issue is not about the "primary" users (by volume). (Remember, 
*traffic volume is not an indicator of importance*.) This is an 
accessibility issue.   Would you consider it acceptable to deny access 
to IETF documents to sight- or hearing-impaired persons because "the 
primary users... don't appear to be disadvantaged"?   If not, why is it 
acceptable to deny access to those who cannot use crypto?

I don't think rsync is an acceptable substitute for FTP (valuable though 
rsync is) for several reasons: (a) it's not that widely known and mostly 
known in the Unix/Linux worlds; (b) people looking for a way to access 
individual files may not stumble on rsync because it's generally a 
mirroring solution; (c) it's not really designed to permit single file 
access even though it can be used that way; (d) a lot of people don't 
know that rsync access to IETF documents even exists or how to use it, 
and (given that http access is already cut off) may have no way of 
discovering it; (e) support for rsync is not integrated into browsers 
(though if browser vendors are also crippling ftp and http URLs that 
point may be moot).

More broadly, I keep seeing examples of people arguing for restrictions 
on how IETF resources may be accessed, based on the assumption that 
/everyone should use IETF resources the way most people use IETF 
resources/.  Trying to force everyone to be alike is deeply offensive 
and counterproductive to IETF's goals.   IETF works best when it has 
wide participation.

Keith

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 18 Nov 2020 04:17:25 -0800

Hi Keith,

> and counterproductive to IETF's goals.   IETF works best when it has 

I think I can accept a sentiment that "not offering FTP access is a
restriction on how IETF resources may be accessed" even though it's a
somewhat convoluted logical exercise to get there.  But what threshold do
you use, then?  It's a restriction on how IETF resources may be accessed
that I can't get them via gopher, too, yet I am not expecting you to jump
up and demand that the IETF set up a gopher server by which I can obtain
RFCs.  Is the only difference between FTP and gopher that we happen to have
been running FTP for some time?

My understanding is that we are at a juncture where staff are migrating
services to a new generation of hardware (or possibly "hardware"; I'm not
very close to the details), and asking whether we need to put in the work
to build support for FTP on the new generation of stuff.  This is not a
"just keep the lights on" question of leaving be what's currently running,
but a question of actively putting in work to build a new thing to provide
the service, and I think it's fair to ask for the cost/benefit analysis to
be done.  You say that "traffic volume is not an indicator of importance",
but what metric can you offer in its stead?  As I see it, Roman and the
tools team have been attempting to run the numbers with the numbers
presently available, and those numbers are at least facially useful
numbers.  (If there were literally zero logged users of the FTP service
over a year, would you be complaining?  What about one?  Where is the
boundary?)  If you can't offer alternative numbers or ways to run the
numbers, I don't think you will be very effective at achieving a different
outcome.  The point we've been getting from Ned and others about
unencrypted access could potentially become one, but I don't think I've
seen it formulated in a very useful fashion (yet).

Thanks,

Ben

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 18 Nov 2020 12:30:29 +0000

> both the "pro" and "anti" side.

^^^ this

In any event, the figures show clearly that ftp usage has declined to 
the point that it's very much a minority thing these days.  Even if the 
IETF continues to provide ftp access for the time being, this decline is 
continuing, so from a consumer point of view, the practical approach 
would be to start thinking about migrating scripts away from it.

Nick

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 18 Nov 2020 13:25:38 +0000

Hi!

> From: ietf <[email-masked]> On Behalf Of Nick Hilliard
> Sent: Wednesday, November 18, 2020 7:30 AM
> To: Theodore Y. Ts'o <[email-masked]>
> Cc: [email-masked]
> In any event, the figures show clearly that ftp usage has declined to the point
> that it's very much a minority thing these days.  Even if the IETF continues to
> provide ftp access for the time being, this decline is continuing, so from a
> consumer point of view, the practical approach would be to start thinking
> about migrating scripts away from it.

... and I'll again extend my offer, if you have a FTP dependent script in production now and need help with the migration to either https:// or rsync:// please reach out to me offline.  Either me or volunteers from home organization will do our best to help you port.

Roman

---

From: Lou Berger <[email-masked]>
Date: Wed, 18 Nov 2020 14:21:03 +0000

Ben

> but I don't think I've seen it formulated in a very useful fashion (yet).

How about: the FTP Service will be discontinued when FTP (RFC959) is moved to Historic status.

Lou

----------
On November 18, 2020 4:17:51 AM Benjamin Kaduk <[email-masked]> wrote:

Hi Keith,

Thanks,

Ben

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 09:22:11 -0500

On 11/18/20 7:17 AM, Benjamin Kaduk wrote:
> restriction on how IETF resources may be accessed"
not merely /how/ they may be accessed (which is important), but also 
/who/ can access them
> even though it's a
> you use, then?

I would say that the broad spectrum of IETF document users should be 
considered, and the decision be made in terms of what is reasonable 
accommodation for that broad spectrum of users.

It doesn't mean you have to support everything forever, but changing the 
access methods without considering how to accommodate that broad 
spectrum of users (and also URLs that will become stale) doesn't seem 
appropriate.

> It's a restriction on how IETF resources may be accessed

There are many differences between FTP and gopher.   FTP was widely used 
for at least a decade before gopher appeared, and gopher was a flash in 
the pan - http eclipsed it almost immediately.   So there was never any 
long-term use of gopher.   A lot of people never even saw gopher before 
they got their hands on a web browser.

> be done.

I haven't seen any cost analysis at all.   The benefit analysis that 
I've seen seems to only care about users in terms of traffic volume, 
which seems to me to be a rather narrow and myopic view.

It's not wrong to ask the question, but the benefits of FTP need to be 
understood and taken seriously rather than dismissed out of hand.    
It's useful to be able to access RFCs and I-Ds as if they were files in 
directories (which they are) and to have a stable API for doing that.  
It's useful to be able to download, or print, many RFCs and I-Ds at once 
as an alternative to having to deal with a web browser's crippled user 
interface.   It's useful to have an alternative to https just in case 
https is blocked by whatever party for whatever reason.  And more 
generally, it's useful to facilitate tool building and to have long-term 
stable and widely-implemented interfaces that those tools can use.

> You say that "traffic volume is not an indicator of importance",
> but what metric can you offer in its stead?

I often find myself thinking that people are too enamored with numbers, 
that people trust numbers because they seem to be precise /even when 
those numbers aren't at all representative of what actually needs to be 
measured./

So my answer is: /maybe stop trying so hard to quantify things/. How do 
you measure the value of one additional IETF participant, one additional 
point-of-view that might shed valuable light on some protocol design 
choices that otherwise would not be shed?   I tend to think that input 
from people who would otherwise be marginalized or excluded can be 
extremely valuable precisely because they can help give the other 
participants a more representative picture of the global Internet 
environment.

Keith

p.s. One kind of answer to whether it's worth it to maintain a service 
might be: how many people are willing to volunteer to fund and perhaps 
maintain it?    If it's important enough to a few people that they're 
willing to fund and run it, maybe that's important enough to keep on.   
I realize that there are inevitably some issues and tensions associated 
with having volunteers run services that IETF participants use.   But 
IETF is fundamentally a volunteer organization, and I believe IETF is 
better at its core mission when the organization actively cooperates 
with and facilitates volunteers - not only at standards development but 
at providing tools and services that improve its ability to function.   
Lately there seems to be organizational hostility toward tool makers, 
though I'm not exactly sure where that's coming from.   IMO the 
organization should see tool making by participants and users as a 
fundamental part of IETF.

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 09:26:32 -0500

> moved to Historic status.
One of the problems I have in general with reclassifying things as 
Historic is that this can lead to deprecation of services that are still 
quite useful, without the implications of such deprecation even being 
discussed, and without providing any advice about how to transition.

"Historic" may be the wrong way of describing the situation. More 
accurately, a protocol may change from mainstream use - where it's the 
widely accepted way of doing some particular thing - to occasional or 
fringe use, where it's still useful in corner cases even though it's no 
longer the widely accepted way of doing that thing.

Keith

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 18 Nov 2020 09:36:57 -0500

There seem to be two groups of users causing the discussion.
One set are folks who use scripts that will be discommoded if we drop 
FTP access.  That is a concern.  But a somewhat manageable one.  And one 
where we havve to at some point be able to say "no, we do not supporting 
things forever".

The other argument is that there exist a set of people who will be 
unable to practically get the documents if we drop the FTP access.  If 
true, that is important.  But we do not appear to ahve any way to 
evaluate the statement as other than a hypothetical.  We know such 
people could exist.  But do we know if they do exist?

It seems to me that arguing for keeping the service because people in 
the second category may exist is a very weak argument.

Yours,
Joel

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 09:47:27 -0500

On 11/18/20 9:36 AM, Joel M. Halpern wrote:

> FTP access.

I don't know where the emphasis on scripts came from.   Certainly there 
are scripts using FTP but that's not the only good reason to use FTP 
even by users who can use https and/or rsync.

Keith

---

From: "Livingood, Jason" <[email-masked]>
Date: Wed, 18 Nov 2020 14:59:52 +0000

One way to surface potential users is perhaps to run an experiment and turn off FTP for 1 week (or whatever period of time) or to repoint the FTP service for that interval to a directory with a single text doc saying we're considering turning off FTP and please email us and tell us about your usage requirements. But this suggestion aside, we do appear to have quite clear usage data to support decision-making.

JL

On 11/18/20, 9:37 AM, "ietf on behalf of Joel M. Halpern" <[email-masked] on behalf of [email-masked]> wrote:

    Yours,
    Joel

---

From: [email-masked]
Date: Wed, 18 Nov 2020 06:42:39 -0800 (PST)

> where we havve to at some point be able to say "no, we do not supporting

I don't think anyone is saying "forever", just "not now". indeed, given that
some of the people making this argument are the same people who believe a total
transition to IPV6 is going to happen - a transition ftp will not survive -
it's pretty obvious this isn't the case.

> unable to practically get the documents if we drop the FTP access.

I was relieved to learn that rsync does not require crypto, so there's at least
one other non-crypto option the IETF still supports. OTOH, the set of
capabilities rsync provides is quite different and much more limited, so this
is not sufficient cause for me to change my position on ftp.

> If true, that is important.  But we do not appear to ahve any way to

And now not only are you asking for information about current conditions we do
not have, you're also asking us to predict the future.

What we do know is that our track record in regards to anticipating unintended
consequences is incredibly poor.

In this regard, there has already been discussion of the downsides to dropping
HTTP support entirely; I see no reason to elaborate further on that.

But if you want a crypto-specific example, you need look no further than how
we're unintentionally introduced widespread non-repudiation into our email
infrastructure.

More specifically, we developed DKIM/DMARC as an anti-phishing measure for
commerical email. It was never intedned to be used for personal email, but
Yahoo deployed it in the personal email space and others have followed suit on
a massive scale. As a result a significant and growing percentage of email is
now signed, to the point where privacy experts are calling for DKIM key release
after rotation to at least partially mitigate the damage we have done.

For me the bottom line is this is a very small cost that at at minimum provides
insurance against some really bad outcomes.

> the second category may exist is a very weak argument.

Right back atcha, Joel.

				Ned

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 18 Nov 2020 15:44:59 +0000

If this is about not rewriting scripts, users could run a cron job to rsync the files to a local repository and then run their own ftp server to run the script.

… or are we about to loose rsync as well?

Stewart

> On 18 Nov 2020, at 14:36, Joel M. Halpern <[email-masked]> wrote:
> 
> One set are folks who use scripts that will be discommoded if we drop FTP access.  That is a concern.  But a somewhat manageable one.  And one where we havve to at some point be able to say "no, we do not supporting things forever".
> 
> The other argument is that there exist a set of people who will be unable to practically get the documents if we drop the FTP access.  If true, that is important.  But we do not appear to ahve any way to evaluate the statement as other than a hypothetical.  We know such people could exist.  But do we know if they do exist?
> 
> It seems to me that arguing for keeping the service because people in the second category may exist is a very weak argument.
> 
> Yours,
> Joel
> 
>> One of the problems I have in general with reclassifying things as Historic is that this can lead to deprecation of services that are still quite useful, without the implications of such deprecation even being discussed, and without providing any advice about how to transition.
>> "Historic" may be the wrong way of describing the situation. More accurately, a protocol may change from mainstream use - where it's the widely accepted way of doing some particular thing - to occasional or fringe use, where it's still useful in corner cases even though it's no longer the widely accepted way of doing that thing.
>> Keith
>

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 18 Nov 2020 19:25:24 +0000

Hi Keith,

From: Keith Moore <[email-masked]>
Sent: Wednesday, November 18, 2020 7:06 AM
To: Roman Danyliw <[email-masked]>; Ned Freed <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: Call for Community Feedback: Retiring IETF FTP Service

The issue is not about the "primary" users (by volume).    (Remember, traffic volume is not an indicator of importance.)   This is an accessibility issue.   Would you consider it acceptable to deny access to IETF documents to sight- or hearing-impaired persons because "the primary users... don't appear to be disadvantaged"?   If not, why is it acceptable to deny access to those who cannot use crypto?

[Roman] The IETF should definitely try to ensure that there is pervasive access to its information.  Toerless asked the same question [1].  He was wondering if current FTP access was bridging access to other communities.

==[ snip ]==

==[ snip ]==

[Roman] I’m no expert in accessibility technology, but what’s the basis to link the “sight- or hearing-impaired persons” population with FTP usage.  Is there accessibility technology of which we are aware that relies on FTP?

[Roman] What accessibility issues is FTP solving?

I don't think rsync is an acceptable substitute for FTP (valuable though rsync is) for several reasons:

[Roman] I wasn’t suggesting that rsync is a substitute of FTP.  Rsync + https can cover FTP use cases (unless using encryption in the https is the problem; however, I’m still unclear on why using https rather than http presents a problem when accessing IETF assets).

(a) it's not that widely known and mostly known in the Unix/Linux worlds;

[Roman] So the belief is that most FTP users are coming from Windows?  Rsync across all platform strikes me a niche.

(b) people looking for a way to access individual files may not stumble on rsync because it's generally a mirroring solution;

[Roman] Agreed.  However, as discussed with Toerless [1] there is little evidence that “stumbling” or as he called it “search/find” is actually happening via FTP.  If this use case of interest, “https://www.ietf.org/ietf-ftp/” provides a very accessible “stumbling” all with a tool (browser) built into all end-user operating systems.  You’ve noted that “traffic volume isn’t importance” but without relying on usage data of how the system is used now we’re reduced to conjuncture and speculation of who our users are and might be.

(c) it's not really designed to permit single file access even though it can be used that way;

[Roman] Agreed.  It would be awkward to use rsync for that.  However, HTTPS seems ideal.

(d) a lot of people don't know that rsync access to IETF documents even exists or how to use it, and (given that http access is already cut off) may have no way of discovering it;

[Roman] Well, they could look at https://www.ietf.org/standards/ids/internet-draft-mirror-sites/ or  https://tools.ietf.org/.

(e) support for rsync is not integrated into browsers (though if browser vendors are also crippling ftp and http URLs that point may be moot).

[Roman] Right, but browsers could use HTTPS to get absolutely everything that is currently in FTP.

Roman

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 14:53:01 -0500

On 11/18/20 2:25 PM, Roman Danyliw wrote:

> Hi Keith,
>
> *From:* Keith Moore <[email-masked]>
> *Sent:* Wednesday, November 18, 2020 7:06 AM
> *To:* Roman Danyliw <[email-masked]>; Ned Freed <[email-masked]>
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: Call for Community Feedback: Retiring IETF FTP Service
>
> primary users... don't appear to be disadvantaged"?   If not, why is 
> it acceptable to deny access to those who cannot use crypto?
>
> [Roman] The IETF should definitely try to ensure that there is 
> pervasive access to its information.  Toerless asked the same question 
> [1].  He was wondering if current FTP access was bridging access to 
> other communities.
>
> ==[ snip ]==
>
> Per the usage data [1], the 85th percentile of traffic comes from 
> entities that don't strongly suggest they would mirror for unique access:
>
It's not the 85th percentile of traffic that you should be looking at.   
It's the remaining 15 percent; the odd uses cases that aren't easily 
characterized.

> [Roman] I’m no expert in accessibility technology, but what’s the 
> basis to link the “sight- or hearing-impaired persons” population with 
> FTP usage.
>
The "accessibility" issue I was referring to was also one that Ned was 
concerned about - are there people who cannot access RFCs and I-Ds 
because they can't use https and therefore crypto?   I don't think rsync 
suffices because it's designed for mirroring rather than file access.   
(Accessibility isn't just about people with physical impairments.)

The reference to sight- or hearing-impaired persons was an analogy.  If 
you don't think they should be prevented from accessing RFCs and I-Ds, 
should those in countries that block https be prevented?   (For 
instance, some countries are currently blocking TLS 1.3 when using ESNI 
because they can't monitor what sites the browsers are talking to.)

> [Roman] I wasn’t suggesting that rsync is a substitute of FTP.  Rsync 
> + https can cover FTP use cases (unless using encryption in the https 
> is the problem; however, I’m still unclear on why using https rather 
> than http presents a problem when accessing IETF assets).
>
no, rsync + https does not cover FTP use cases.
> [Roman] So the belief is that most FTP users are coming from Windows?  
> Rsync across all platform strikes me a niche.
>
Yes, I think rsync is a niche...that is the point I was making.
>
> (b) people looking for a way to access individual files may not 
> stumble on rsync because it's generally a mirroring solution;
>
> [Roman] Agreed.  However, as discussed with Toerless [1] there is 
> little evidence that “stumbling” or as he called it “search/find” is 
> actually happening via FTP.  If this use case of interest, 
> “https://www.ietf.org/ietf-ftp/” provides a very accessible 
> “stumbling” all with a tool (browser) built into all end-user 
> operating systems.
>
I wasn't referring to finding IETF content via stumbling, I was instead 
asking: how does a user or service who needs a better solution than 
https find out about rsync access?

> You’ve noted that “traffic volume isn’t importance” but without 
> relying on usage data of how the system is used now we’re reduced to 
> conjuncture and speculation of who our users are and might be.

The same is true when "relying on usage data" alone, there's a lot of 
conjecture and speculation about what that usage data means, especially 
for the usage data that doesn't easily cluster.

> [Roman] Agreed.  It would be awkward to use rsync for that.  However, 
How many times do I have to keep pointing out that a web browser is a 
really lousy tool for trying to access multiple RFCs or drafts at the 
same time?   I can't tell you how many hours I've wasted trying to deal 
with web interfaces to document that required me to individually click 
on each one, download each one, specify a destination for each one, and 
keep track of which ones actually got downloaded, all before I could do 
anything with that set of documents... and wondered "why can't I just 
ftp this stuff in one command?"   Web browsers suck for file 
downloading, and http (without WebDAV) doesn't provide enough protocol 
hooks to build a decent client that would support that.   (Yes this is 
still "individual file access" as compared to syncing the whole server.)
>
> (d) a lot of people don't know that rsync access to IETF documents 
> even exists or how to use it, and (given that http access is already 
> cut off) may have no way of discovering it;
>
> [Roman] Well, they could look at 
> https://www.ietf.org/standards/ids/internet-draft-mirror-sites/ or 
> https://tools.ietf.org/.
>
How are they going to find those if they can't use https?
>
> (e) support for rsync is not integrated into browsers (though if 
> browser vendors are also crippling ftp and http URLs that point may be 
> moot).
>
Arrgh.   I give up.  Clearly you've never actually had to do this kind 
of thing.   But the idea that "everybody should use the Internet like I 
do" seems out of place for IETF.

Keith

---

From: "John Levine" <[email-masked]>
Date: 18 Nov 2020 15:36:18 -0500

>How about: the FTP Service will be discontinued when FTP (RFC959) is moved to Historic status.

In that case, I want my gopher access back.

Jason's suggestion to point the FTP server at a stub with a README for
a week seems reasonable. I would be surprised if anyone other than
Keith conmplained.

R's,
John

---

From: Roman Danyliw <[email-masked]>
Date: Wed, 18 Nov 2020 21:17:58 +0000

Hi!

Sent: Wednesday, November 18, 2020 2:53 PM

From: Keith Moore <[email-masked]><mailto:[email-masked]>
Sent: Wednesday, November 18, 2020 7:06 AM
To: Roman Danyliw <[email-masked]><mailto:[email-masked]>; Ned Freed <[email-masked]><mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: Call for Community Feedback: Retiring IETF FTP Service

==[ snip ]==

It's not the 85th percentile of traffic that you should be looking at.   It's the remaining 15 percent; the odd uses cases that aren't easily characterized.

[Roman] I’m no expert in accessibility technology, but what’s the basis to link the “sight- or hearing-impaired persons” population with FTP usage.

The "accessibility" issue I was referring to was also one that Ned was concerned about - are there people who cannot access RFCs and I-Ds because they can't use https and therefore crypto?   I don't think rsync suffices because it's designed for mirroring rather than file access.   (Accessibility isn't just about people with physical impairments.)

The reference to sight- or hearing-impaired persons was an analogy.  If you don't think they should be prevented from accessing RFCs and I-Ds, should those in countries that block https be prevented?   (For instance, some countries are currently blocking TLS 1.3 when using ESNI because they can't monitor what sites the browsers are talking to.)

[Roman] In case there is concern about the TLS configuration on www.ietf.org<http://www.ietf.org>, it is quite permissive to ensure flexibility .  See https://www.ssllabs.com/ssltest/analyze.html?d=www.ietf.org&s=104.16.45.99&hideResults=on.  TLS v1.0 – 1.3 is supported.  Likewise, the ciphersuites are extremely generous.

Roman

---

From: "John Levine" <[email-masked]>
Date: 18 Nov 2020 16:19:36 -0500

>after rotation to at least partially mitigate the damage we have done.

Urrgh. We correctly expected DKIM to be used for all sorts of mail,
but without expecting the DKIM domain to match the From (other than
the experimental and unused ADSP extension.) DMARC made "aligned"
signatures treated specially, but the signatures didn't change.

What we didn't anticipate is that large mail systems would never
rotate their keys and use the same DKIM signing key for many years, so
you can easily check old messages with old signatures. I suppose it is
kind of a surprise that people use them for non-repudiation, but since
the signatures aren't technically very different from S/MIME or PGP
signatures, it shouldn't be that surprising.

R's,
John

---

From: [email-masked]
Date: Wed, 18 Nov 2020 13:26:22 -0800 (PST)

> signatures, it shouldn't be that surprising.

FWIW, I actually pointed out this and other potential downsides of using
signatures way back when DKIM was originally standardized, but (a) We don't
have an alternative technology, or even an idea of what an alternative
technology would be, and (b) As you say, we didn't expect it to be this bad.

There's also no way to fully mitigate the issue: Someone can always immediately
apply an independent timestamping service to every message they see, making
subsequent exposure of the private key meaningless.

That said, a mechanism for publishing/expiring DKIM private keys is something
the IETF might want to standardize.

				Ned

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 13:44:12 -0800

> signatures, it shouldn't be that surprising.

It was certainly our intention that it was at least for enterprise since 
that's the use case we were most interested in at Cisco. But Ned is 
right that a lot of our motivation at Cisco was driven by spear 
phishing. We didn't ultimately succeed because there were just too many 
things emitting mail in closets from 386 servers everybody was afraid to 
turn off. I hope it's a different situation now after 15 years.

The funny thing about this non-repudiation issue is that I don't recall 
anybody bringing it up, and that's probably because it was a non-issue 
then because submission authentication was pretty rare. DKIM couldn't 
prove anything beyond that it was the domain that sent it which is 
pretty ho-hum for say a gmail. Only changing policies about submission 
authentication closed the loop. I've always wondered whether DKIM had a 
part in that policy change, or whether it was just in the water to clean 
up lax email provider policies.

I frankly see non-repudiation as an unexpected benefit. The internet is 
forever. Film at 11.

Mike

---

From: "John R Levine" <[email-masked]>
Date: 18 Nov 2020 16:45:19 -0500

> the IETF might want to standardize.

I've started to publish my old private keys since I rotate every month but 
I agree a standard way to tell people where to look would be nice.

Regards,

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 13:48:16 -0800

> the IETF might want to standardize.
I really don't see the point of it. Mail providers' motivations are very 
different than individual users. I don't see how you can align those 
motivations easily to meet whatever the goal is.

Mike

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 13:51:12 -0800

On 11/18/20 1:45 PM, John R Levine wrote:
> I've started to publish my old private keys since I rotate every month 
> but I agree a standard way to tell people where to look would be nice.
>
Why isn't just deleting/replacing the selector sufficient? It's not as 
definitive but it's a lot simpler.

Mike

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 18 Nov 2020 22:04:57 +0000

Hiya,

> definitive but it's a lot simpler.

Publishing the private key enables various forms of
denyability - if someone claims msg1 is original
anyone with access to the private can produce a
msg2 that seems as cryptographically correct but
is clearly bogus (e.g. containing lottery numbers
that post-date message timestamps).

Yes an adversary could have gotten an independent
signed timestamp on msg1 before the private was
published but that seems low probability.

I'd support development of such a standard if it
had a good chance of deployment as I think it'd
also encourage key rotation.

S.

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 17:12:34 -0500

Sent from my iPhone

> On Nov 18, 2020, at 3:37 PM, John Levine <[email-masked]> wrote:
> 
> a week seems reasonable.

Because it is perfectly OK to break a non-interactive service with a message that chances are good no one will ever see.  But when people realize the service is broken and turn their clients off, somebody will say “see, nobody was using this after all!”

It reminds me of the algorithm for determining whether someone’s a witch- throw them into a large body of water and see if they float.

Keith

BTW, John, you are out of line.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 14:17:17 -0800

On 11/18/20 2:04 PM, Stephen Farrell wrote:
>>> I've started to publish my old private keys since I rotate every 
>>> month but I agree a standard way to tell people where to look would 
>>> be nice.
>>>
>> Why isn't just deleting/replacing the selector sufficient? It's not 
>> as definitive but it's a lot simpler.
>

Yes, i acknowledge that above albeit obliquely. What i don't see is how 
you align providers goals' with individual users' goals.

> published but that seems low probability.
It really depends on the worth of the data, right? LEA would certainly 
do such a thing if they were serveilling somebody.
> also encourage key rotation.
I forget who said that they were surprised about lack of key rotation, 
but color me completely unsurprised. This is just inertia 101. Maybe 
large ESP's might get around to automating key rotation, but for the 
vast majority enterprise this is going to be pretty low down the 
priority list, and more likely an anti-goal as tracking whether their 
employees are misbehaving is a feature not a bug.

Mike

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 18 Nov 2020 22:29:53 +0000

> you align providers goals' with individual users' goals.

My guess is that email service providers that are
concerned about potential leakage of message store
content would be motivated to do this so as to
re-assure their users and/or maybe help avoid future
liability (financial or moral).

> do such a thing if they were serveilling somebody.

Sure. The main adversary I had in mind for this mechanism
would be a message (store) leaker, not LEAs. I guess the
idea could be ineffective or even counterproductive
with other threat models, not sure - but would need
checking out, for sure.

> employees are misbehaving is a feature not a bug.

Agreed. If, however, publishing old private keys is seen
as a way to reduce potential liability, then that might
motivate rotation. (That's my guess, but only a guess.)

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 18 Nov 2020 22:32:44 +0000

Hiya,

> but I agree a standard way to tell people where to look would be nice.

Where are you publishing 'em?

Ta,
S.

---

From: John E Drake <[email-masked]>
Date: Wed, 18 Nov 2020 22:34:36 +0000

Hi,

Is there any way you could summarize your perspective in a single email and then stop?

Yours Irrespectively,

John

Juniper Business Use Only

> From: ietf <[email-masked]> On Behalf Of Keith Moore
> Sent: Wednesday, November 18, 2020 5:13 PM
> To: John Levine <[email-masked]>
> Cc: [email-masked]
> [External Email. Be cautious of content]
> 
> 
> Sent from my iPhone
> 
> Because it is perfectly OK to break a non-interactive service with a message that
> chances are good no one will ever see.  But when people realize the service is
> broken and turn their clients off, somebody will say “see, nobody was using this
> after all!”
> 
> It reminds me of the algorithm for determining whether someone’s a witch-
> throw them into a large body of water and see if they float.
> 
> Keith
>

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 14:41:25 -0800

On 11/18/20 2:29 PM, Stephen Farrell wrote:
>> Yes, i acknowledge that above albeit obliquely. What i don't see is 
>> how you align providers goals' with individual users' goals.
>

It would be pretty disasterous regardless of a valid DKIM signature. 
Most people have no clue that email *also* prevents deniability but  the 
damage would already be done because nobody's going believe that 
somebody's long cheating email romance was just elaborately spoofed. 
Same goes for providers if they screw up: an invalidated DKIM signature 
is not going to protect them from lawsuits.

MIke

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 14:43:31 -0800

On 11/18/20 2:32 PM, Stephen Farrell wrote:
>> month but 
The easiest that i can think of is just putting it in the selector itself.

Mike

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 17:43:31 -0500

On 11/18/20 4:17 PM, Roman Danyliw wrote:

> [Roman] In case there is concern about the TLS configuration on 
> www.ietf.org <http://www.ietf.org>, it is quite permissive to ensure 
> flexibility .  See 
> https://www.ssllabs.com/ssltest/analyze.html?d=www.ietf.org&s=104.16.45.99&hideResults=on. 
> TLS v1.0 – 1.3 is supported.  Likewise, the ciphersuites are extremely 
> generous.

Well, I'm glad for that at least.

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 18 Nov 2020 22:44:48 +0000

Hiya,

On 18/11/2020 22:41, Michael Thomas wrote:
> Most people have no clue that email *also* prevents deniability butÂ  
> the damage would already be done because nobody's going believe that 

Maybe or maybe not. In the case of the DNC/Podesta it might
have had utility for someone wishing to claim forgery. I
don't really claim to know whether it'd be a useful legal
mechanism or not, so while I do think it might, that'd need
more checking for sure.

S.

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Nov 2020 17:53:20 -0500

> Is there any way you could summarize your perspective in a single email and then stop?

I can try.

1. Discontinuing FTP service will be disruptive no matter what. Broken 
scripts, broken links, tools that people use will no longer work.

2. Discontinuing FTP without replacing it with something equivalent 
(e.g. a file-access service that can list directories, permit tree 
walking, distinguish ordinary files from directories from symlinks in 
addition to permitting downloading) will rob IETF document users of 
valuable functionality that is not provided by either web browsers or rsync.

3. WebDAV appears to be a functional equivalent to FTP for this 
purpose.   However, replacing FTP with WebDAV looks like replacing a 
mature and well-established protocol that enjoys very widespread client 
support, with a less mature, less widely-supported, and even more 
baroque protocol than FTP, in addition to being disruptive.   Though it 
would permit encryption, so that's a plus for WebDAV.

(In a brief search I did not find evidence that some countries are 
[still?] blocking https, only that some countries are blocking TLS 1.3 + 
ESNI. )

4. I would be surprised if the opex associated with WebDAV were actually 
less than running an FTP server, but it's possible.

IMO continuing to support FTP is a better decision than migrating to 
WebDAV, but either of those alternatives is better than simply pulling 
the plug on FTP.

Keith

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 14:56:05 -0800

On 11/18/20 2:44 PM, Stephen Farrell wrote:
>> Same goes for providers if they screw up: an invalidated DKIM 
>> signature is not going to protect them from lawsuits.
>
Meta: does anybody actually know what was scandalous about her emails?

But the recent idiocy from Rudy with Hunter's supposed email would have 
been nice to repudiate him. It didn't actually need it because the 
keystone kops were so thoroughly incompetent, but one day this sort of 
thing could come in really handy. Given that the internet is forever on 
so many other levels, publishing private keys seems too little, too late.

Mike

---

From: "John Levine" <[email-masked]>
Date: 18 Nov 2020 18:49:41 -0500

>turn off. I hope it's a different situation now after 15 years.

DMARC includes a reporting feature you can turn on without turning on
any of the policy stuff. It's exactly so you can find those servers in
closets. Cisco now publishes a p=quarantine DMARC policy which
suggests they think their random server problem is under control.

>pretty ho-hum for say a gmail.

Large webmail systems have always been pretty strict about what header
addresses you can use. I don't think it was ever easy for one Gmail
user to send mail pretending to be another.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 16:09:52 -0800

> suggests they think their random server problem is under control.

The larger problem we had was that Cisco did lots of acquisitions which 
made it really hard to know what we were up against. I like the 
reporting feature, but the rest of it looks like warmed over adsp to me.

> user to send mail pretending to be another.
But it was turning on submission auth that makes a really good case that 
a person did in fact send that piece of email. I wonder if this has been 
used legally yet? Most likely the vast majority of the time it doesn't 
need to come down to that.

Mike

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 19 Nov 2020 00:23:10 +0000

Hiya,

On 18/11/2020 22:56, Michael Thomas wrote:
> Given that the internet is forever on 

I think your conclusion there is a thing where reasonable
people can disagree as to possible outcomes.

I don't think it's too late if there is some sufficient new
benefit accruing to a private key publisher. And that may be
the case as already discussed.

I don't think it's too little either - since everything else
in a leaked mail or message store is freely malleable, access
to the relevant private key is arguably not too little.

There's still plenty of possibility for this not getting
used, but I think it worth a bit of effort to try find
that out. (And, I admit, part of me likes the idea of
publishing private keys for virtuous reasons:-)

S.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 16:31:22 -0800

> publishing private keys for virtuous reasons:-)
Obviously I'm not vehemently against this -- people can do whatever they 
like with their private keys -- but it seems in order to do this 
properly it's a tiny bit protocol specification and a lot of bit of BCP 
describing the problem space of who, when, and why somebody would do this.

Mike

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 19 Nov 2020 00:34:25 +0000

> describing the problem space of who, when, and why somebody would do this.

That's fair. It'd also require a bit of analysis as to how
published private keys could be used for attempted mischief
as well.

My take-away is I'll add doing this for some DKIM keys to
my to-do list.

S.

---

From: [email-masked]
Date: Wed, 18 Nov 2020 16:21:59 -0800 (PST)

> need to come down to that.

I can't speak to use in court, but this seems relevant:

  https://blog.erratasec.com/2016/10/yes-we-can-validate-wikileaks-emails.html#.WA4khPkrLAW

Also, credit where credit is due: Matthew Green is the primary impetus
behind the push to publish keys:

 https://blog.cryptographyengineering.com/2020/11/16/ok-google-please-publish-your-dkim-secret-keys/

				Ned

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Nov 2020 19:31:39 -0800

On 11/18/20 4:34 PM, Stephen Farrell wrote:
>> Obviously I'm not vehemently against this -- people can do whatever 
>> they like with their private keys -- but it seems in order to do this 

So I just discussed this with lawyer friend of mine who would actually 
be impacted by this, and suffice it to say this could get very 
complicated in an actual legal setting. I also found out that i could 
have a gig as an expert witness. I'm weighing whether this a good thing 
or a bad thing :)

Mike

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 Nov 2020 05:04:47 +0100

> would permit encryption, so that's a plus for WebDAV.

"more baroque"? Can you elaborate?

> less than running an FTP server, but it's possible.

For some http servers, WebDAV support comes as module you just need to
enable. So no additional server at all.

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Thu, 19 Nov 2020 12:40:17 +0000

>> TLS v1.0 – 1.3 is supported.  Likewise, the ciphersuites are extremely

That is true for the main IETF site but not for the tools sites which is 
what I imagine that almost everyone here will be concerned with.

Tom Petch

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Nov 2020 10:42:23 -0500

> "more baroque"? Can you elaborate?

The number of features WebDAV tries to support makes FTP look simple, 
especially when using it for the purpose of granting file-level access 
to a read-only repository.

> enable. So no additional server at all.

Well, if you're running your own servers, opex isn't a linear multiple 
of the number of servers you have to run.

(Though admittedly if you're outsourcing those servers to a CDN, as IETF 
apparently is with www.ietf.org, that depends on the CDN's charging 
model.  I can understand why IETF would use a CDN - it definitely helps 
manage some kinds of operational risk - though it might be overkill for 
an FTP server that could probably run reasonably well on a Raspberry Pi 4.)

Keith

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 Nov 2020 17:00:15 +0100

> to a read-only repository.

Not sure what features you are referring to, but many things are
optional. For read-only access, you basically just need PROPFIND (aka,
"ls") and bit of OPTIONS (if the client insists to do discovery).

Best regards, Julian

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 19 Nov 2020 17:24:44 +0000

Hi Tom!

> From: tom petch <[email-masked]>
> Sent: Thursday, November 19, 2020 7:40 AM
> To: Keith Moore <[email-masked]>; Roman Danyliw
> <[email-masked]>; Ned Freed <[email-masked]>
> Subject: TLS access Re: Call for Community Feedback: Retiring IETF FTP Service
> 
> >> TLS v1.0 – 1.3 is supported.  Likewise, the ciphersuites are
> >> extremely generous.
> 
> That is true for the main IETF site but not for the tools sites which is what I
> imagine that almost everyone here will be concerned with.

Understood.  The good news is that tools.ietf is currently configured for TLS 1.0 - 1.2.  FWIW, datatracker.ietf and www.ietf are the canonical repositories for the I-Ds and RFCs in the IETF.

The full answer is that documents over HTTPS come from 3 domains and 6 formats (see the buttons next to "format" on the status tab of datatracker page):

"plain text" = https://www.ietf.org/archive/id/*.txt
"xml" = https://www.ietf.org/archive/id/*.xml
"pdf" = https://tools.ietf.org/pdf/*.pdf
"htmlized (tools) = https://tools.ietf.org/html/*
"htmlize" = https://datatracker.ietf.org/doc/html/
native datatracker txt = https://datatracker.ietf.org/doc/*

Per those 3 domains, the config is as follows:
-- datatracker.ietf.org = TLS 1.0 - 1.2
-- www.ietf.org = TLS 1.0 - 1.3
-- tools.ietf.org. = TLS 1.0 - 1.2

Bottom line, if you have a browser or https library that implemented state-of-the-art as of 1999 (the publication date of TLS 1.0, RFC2246), you can reach all three domains over https -- so circa Internet Explorer v5 on Windows 98 SE, before MacOS, or Netscape 6.

Since we're talking about these different domains, the original proposal [1]  and the follow-up stats [2] said that HTTPS usage is being undercount.  All of the prepared stats only counted traffic to tools.ietf.org, not datatracker or www.ietf.  Therefore FTP traffic is an even smaller fraction of the overall access than previously discussed.

Regards,
Roman

[1] https://www.ietf.org/media/documents/Retiring_IETF_FTP_Service.pdf
[2] https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJsHI4uyDlQ2dHip8jo/edit

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 Nov 2020 10:09:35 -0800

On 11/18/20 8:08 PM, John R. Levine wrote:
>> But it was turning on submission auth that makes a really good case 
>> that a person did in fact send that piece of email. ...
>
> Gmail, Yahoo, and the like have always required that users 
> authenticate before sending mail.  Otherwise they'd be open relays.

There were tons of open sewers back then. The ISP's were especially bad. 
Gmail wasn't even publicly available until 2004.

>
>> I wonder if this has been used legally yet?
>
> Not that I ever heard.  In court cases they tend to be more interested 
> in chain of custody than technical features.
>

I pinged a lawyer friend last night who deals in an area that is 
affected by email non-repudiation and explained this entire thing to him 
(no small thing because he's not very technical), and his ears 
definitely perked up. He did say when somebody did dispute they actually 
sent a piece of mail, they'd call in an "email expert" witness who would 
walk them through why it wasn't forged. I have no idea if they resort to 
using DKIM as one of their arguments, i'm guessing not because the 
entire idea of forgery with all of the other evidence probably makes it 
pretty far fetched. But if there's enough money on the line...

Mike

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 19 Nov 2020 19:40:18 +0000

>  FWIW, datatracker.ietf and www.ietf are the canonical repositories for the I-Ds and RFCs in the IETF.

Separate question: we were told for the newcomers slides to use rfc-editor.org for official RFC references.  How do we resolve that difference of views?

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Nov 2020 15:01:18 -0500

On 11/19/20 1:09 PM, Michael Thomas wrote:

> He did say when somebody did dispute they actually sent a piece of 
> mail, they'd call in an "email expert" witness who would walk them 
> through why it wasn't forged. 

One of my hats.

> I have no idea if they resort to using DKIM as one of their arguments, 
> i'm guessing not because the entire idea of forgery with all of the 
> other evidence probably makes it pretty far fetched.

Absolutely I would "resort" to such, though I hope I'm never asked to 
support some irresponsible or frivolous action.   I would use every 
shred of evidence I could find.

I do understand why having a MSP provide a free non-optional 
non-repudiation service is not a great thing in general, and think that 
disclosing old private keys is probably a good way to remedy that.  
(just make sure that the repository of old private keys is very well 
advertised).

But there are lots of legitimate, responsible reasons for validating 
that some particular old message is authentic.

(To date I've never been asked to make a case for authenticity of a 
message old enough to benefit from DKIM.)

Keith

---

From: [email-masked]
Date: Thu, 19 Nov 2020 11:58:22 -0800 (PST)

> Separate question: we were told for the newcomers slides to use
> rfc-editor.org for official RFC references.  How do we resolve that difference
> of views?

FWIW, I think that's a good choice for someone not participating in the IETF,
but a suboptimal one for someone who is, or will be.

The nice thing about datatracker.ietf.org is how easy it is to get to related
resources - stuff IETFers need to see, but non-IETFers generally do not.

I do find it odd that DuckDuckGo seems to prefer tools.ietf.org (fine, I
guess), and datatracker.ietf.org (?) and even internetsociety.org (??) to
rfc-editor.org, but it's not like we have any control over that.

				Ned

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 Nov 2020 12:17:23 -0800

>> through why it wasn't forged. 
Who knew there would be a cottage industry for this. Huh.
>
>> I have no idea if they resort to using DKIM as one of their 
>> arguments, i'm guessing not because the entire idea of forgery with 
>> all of the other evidence probably makes it pretty far fetched.
>
> non-repudiation service is not a great thing in general, and think 
> that disclosing old private keys is probably a good way to remedy 
> that.  (just make sure that the repository of old private keys is very 
> well advertised).
>

Sure, there are pluses and minuses. It's why I think the real work of 
publishing keys is in the BCP aspect of it. User's and provider's goals 
are not very well aligned.

I'm still getting over the shock that DKIM played a big role in But Her 
Emails that Ned linked to.

Mike

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 Nov 2020 15:00:18 -0800

> My take-away is I'll add doing this for some DKIM keys to
It occurs to me that it doesn't even need to be a one-size fits all 
solution for providers. All a provider would need to do is have an opt 
in for users to to use selectors who are regularly repudiated, and keep 
the current non-repudiation selectors forever.

If you were to pursue this, what venue would you use? Isn't everything 
shut down?

Mike

---

From: "John Levine" <[email-masked]>
Date: 19 Nov 2020 18:50:06 -0500

>the current non-repudiation selectors forever.

Sounds like an excellent way to get users to say huh? What? I would be
amazed if one user in 100 ever thought about whether their mail might
be verifiable several years from now and what that might imply.

Personally, I rotate every month just so the amount of mail at risk if
I screw up the key management is limited.

$ host -t txt k2011._domainkey.examp1e.com
k2011._domainkey.examp1e.com descriptive text "v=DKIM1; h=sha256; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3Jbgcz4g8viVSkByv4CwRY8RQ4K5Q2ksPeUnsuQKfdLXAUa3/KkHBCLYLkc+Vw3jBDzeispPGSPmUG2KbttxCEV7+CxAGUinEAjaX7GHsw7viOI8ZXxDnbTNlrFygal0OybB0Yk63bBTHyIKgwvvRX/VJVz7mvV9iESzADWIgJ26vugfUUDbr/zPaOynb8" "lBaT8ZShkX8OTRoqBNlsdq4chQWxDatjsMJLFxYT1p8uG3+iT8U8wcSHlDGJD1NF9+xD+yNWtUJuj6d3vkN04yv4gUEc7CHog84NOjaYmDSaoJQ/p3/zkvwOfJdTAldeAPJ7Smj7KVg9exK+D8aiIqjQIDAQAB ;" "n=For=20private=20key=20see=20https://www.iecc.com/dkimkey/k2011/ ;"

R's,
John
[email-masked]

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 Nov 2020 16:06:59 -0800

> be verifiable several years from now and what that might imply.

Yes, but the same can said be for the repudiation crowd too. There is no 
one-size fits all for this problem. But giving the tin-foil hat types 
and security geeks an opt in, they would probably find each other 
relatively easily. I find it dubious that providers are not going to 
find this anything but a big yawn, honestly. Dealing with intractable 
problems for no business benefit is usually not a good career path move.

Mike

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 20 Nov 2020 10:24:02 +0000

Hi Stewart!

> From: ietf <[email-masked]> On Behalf Of Stewart Bryant
> Sent: Wednesday, November 18, 2020 10:45 AM
> To: Joel Halpern <[email-masked]>

[snip]

> … or are we about to loose rsync as well?

To reassure you (as the IESG Tool Rep), there are no plans to transition away from rsync.

Roman

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 20 Nov 2020 10:42:12 +0000

Hi Rich!

> From: Salz, Rich <[email-masked]>
> Sent: Thursday, November 19, 2020 2:40 PM
> Subject: Re: TLS access Re: Call for Community Feedback: Retiring IETF FTP
> Service
> 
> Separate question: we were told for the newcomers slides to use rfc-editor.org
> for official RFC references.  How do we resolve that difference of views?

I'm not knowledgeable enough to be the authority on this.

I used "canonical" in a sloppy way to convey that the datatracker/www-ietf is authoratative when compared to tools.ietf.

Roman

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 20 Nov 2020 11:31:58 +0000

Hi!

We are about half through the feedback period and helpful discussion has already occurred.  Thank you for the input.  

To help focus further discussion and to help onboard those that have not weighed in yet, a summary of the thread to date can be found at https://github.com/ietf/ftp-retirement-consult-2020/blob/main/status.md

Regards,
Roman

---

From: Chris Inacio <[email-masked]>
Date: Fri, 20 Nov 2020 16:38:34 +0000

+1.

FTP was great in its day.  I also really liked gopher.  Maybe more than the current web.  But let’s be real - that’s just not practical at this point, IMHO.  The Internet (at large) has moved on, possible even enabled by IETF, we should keep to that reality.

Chris

> On Nov 16, 2020, at 5:43 AM, Russ Housley <[email-masked]> wrote:
>

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 20 Nov 2020 17:10:40 +0000

>    I'm not knowledgeable enough to be the authority on this.

Fair enough.  Let's chase this down tho and figure out what it should be.  "There  can be only one" definitive source.

---

From: Keith Moore <[email-masked]>
Date: Fri, 20 Nov 2020 12:17:03 -0500

> Fair enough.  Let's chase this down tho and figure out what it should be.  "There  can be only one" definitive source.

It seems clear that rfc-editor.org is the authoritative source for RFCs; 
they're the ones who publish RFCs after all.

IETF's mirror of the RFC Editor's repository/archive is just for 
convenience.

Keith

---

From: tom petch <[email-masked]>
Date: Fri, 20 Nov 2020 17:18:53 +0000

> Bottom line, if you have a browser or https library that implemented state-of-the-art as of 1999 (the publication date of TLS 1.0, RFC2246), you can reach all three domains over https -- so circa I…

Well, no, not in my experience.
I originally configured Windows 98SE for TLS1.0 with certificate 
checking and can today access IANA, ISOC, ITU-T, IEEE, Metropolitan 
Opera inter alia, at least to the home pages, but have 100% failure on 
anything ietf.org - along with Cisco, IBM, RIPE, Microsoft, all of which 
used to work - and so was forced to stop using that several years ago, 
probably at the time of the announcement that in future all ietf access 
would be https:  My logs do not go back that far and anything I look at 
promptly updates last accessed so I cannot see when I last accessed e.g. 
an IETF favorite successfully on 98SE.  (I do not see later Windows as a 
better user interface for contributing to the IETF, i.e. downloading I-D 
and RFC, reviewing and commenting and so did not leave 98SE until I was 
forced to; software currency has not been a concern).

I then used Windows XP until again, access to the IETF web pages stopped 
working, probably 2016, and that remains the case today.  It also is 
configured with TLS 1.0 (not SSL2, SSL3) with certificate checking.  I 
get no prompts about certificate failures which I get plenty of with 
other web sites.  I can access Cisco, Amazon and Google in addition to 
those listed above but not IBM, Microsoft, RIPE.

Your six URL above are interesting.  /pdf/ on Windows XP says no such 
directory so clearly tools.ietf.org is accessible with Windows XP, but 
www. and datatracker. both fail with the usual error message, 'Internet 
Explorer cannot display the page', as does [1] below, and this is what I 
get for any failure to any other organisation, be it no DNS, RJ45 
unplugged, http://192.168.254.1 (.1.254 works to the ADSL modem), almost 
any error except a certificate failure or a 404 so while the issue could 
be with the browser, I think not.  When 98SE failed, I assumed that the 
ietf was demanding TLS 1.1 but from what you say, I suspect it is 
ciphersuite.  XP has all updates applied as of end-of-currency, 98SE 
less so.  Both only go up to TLS 1.0.

The software is out of the box, shrink-wrapped, no fancy add-ons, just 
Acrobat Reader; what else does one need to contribute to the IETF apart 
from access to the ietf website:-)

Tom Petch

---

From: Keith Moore <[email-masked]>
Date: Fri, 20 Nov 2020 12:27:29 -0500

> FTP was great in its day.  I also really liked gopher.  Maybe more than the current web.  But let’s be real - that’s just not practical at this point, IMHO.  The Internet (at large) has moved on, p…

I don't want to single you out specifically, Chris, but I really think 
this is a mischaracterization that lots of people are buying into.

It's not the case that some people choose FTP over HTTP and others 
choose HTTP over FTP.   Rather, some people find tools that use FTP more 
useful than web browsers, /for some purposes/. Even the people who use 
FTP sometimes probably use web browsers for most RFC access.  But for 
some purposes, FTP is a better choice, because it supports functionality 
that web browsers don't support and can't be supported with vanilla HTTP.

So "the Internet" has /not/ moved on, but rather, some people use better 
tools than others.

The argument to deprecate FTP is essentially an argument that people who 
/prefer to use better tools shouldn't be able to do so/.  It's an 
argument to cripple those tools and impair those people's ability to work.

Of course there is some cost to maintaining an FTP server, but it should 
not be a huge cost.  There's certainly not been any support for an 
argument that "that's just not practical".  Has there even been an 
estimate of the monetary cost of maintaining FTP?

And "let's be real" sounds like an argument for a certain kind of 
prejudice - it could be rephrased as "let's force everybody to conform 
to some completely arbitrary fashion".

I don't expect IETF to support forever every possible tool that everyone 
might want to use.   But there's been no sound argument given so far 
that it should cripple FTP-based tools.

Keith

---

From: Russ Housley <[email-masked]>
Date: Sat, 21 Nov 2020 15:43:00 -0500

> On Nov 20, 2020, at 12:10 PM, Salz, Rich <rsalz=[email-masked]> wrote:
> 

The definitive source is rfc-editor.org.  The files are copied from there for serving by ietf.org.

Russ

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 24 Nov 2020 10:30:21 -0700

It’s always been a bit frustrating to get FTP to wok well with all NATs. Definitely time to get rid of it and encourage people to use better things. If there is a real need for this, some other group can mirror things onto something that provides FTP. 

> On Nov 9, 2020, at 7:23 PM, Roman Danyliw <[email-masked]> wrote:
> 
> Hi!
>

---

From: John C Klensin <[email-masked]>
Date: Tue, 24 Nov 2020 17:05:07 -0500

--On Friday, November 20, 2020 11:31 +0000 Roman Danyliw
<[email-masked]> wrote:

> Hi!
> 
> We are about half through the feedback period and helpful
> discussion has already occurred.  Thank you for the input.  
> 
> To help focus further discussion and to help onboard those
> that have not weighed in yet, a summary of the thread to date
> can be found at
> https://github.com/ietf/ftp-retirement-consult-2020/blob/main/
> status.md

Roman,

Thank you for your note and the pointer to the summary.  I
largely agree with Keith's comments today in
https://mailarchive.ietf.org/arch/msg/ietf/ZnI-i5sB-d9eq7KDfkFB3Gb5ZPc
but want to add a few things.

First, it may be a trivial detail, but you undercounted those
who objected.  I'm certainly on that list unless the criterion
for being counted is posting mostly-redundant arguments over and
over again.  That, however, leads to the non-trivial part and
one of those "everything is connected to everything else"
problems.  

I have gone back and checked through my tools and I have been
using FTP access to I-Ds very little in the last few years [1].
I do use FTP to access RFCs, but I do so on the RFC Editor site,
not the IETF one, both because some of my tools are old enough
that the IETF was not maintaining an RFC repository and because
the IETF does not seem to understand the importance of stable
locations or links.  So, in addition to not showing up in the
summary list of those who objected, I don't show up if you are
monitoring use of FTP access to IETF materials.  But I am
opposed to making this change nonetheless.

So I would like to take a step back from most of the details of
the discussion and talk about where it seems to come from.
Keith wrote:

	'And "let's be real" sounds like an argument for a
	certain kind of prejudice - it could be rephrased as
	"let's force everybody to conform to some completely
	arbitrary fashion".'

I see the comments about encryption in the summary under "FTP
provides a unique interface (Use Cases)" in the summary (and a
great deal of traffic in this thread) as being about that same
problem.  

So, again, let's take a step back, noting that this gets back to
my comments about a slippery slope between "drop FTP access" and
"ban everything but HTTPS".   Am I in favor of making encrypted
access to Internet resources available and having them work well
from the user's standpoint as well as securely?  Absolutely.  Am
I in favor of educating users as to why encryption is good for
them and they should prefer it when they have the option?  Yes
again, although I'm not sure that is the IETF's job.  But
neither of those imply a stance that feels equivalent to "if you
want to use unencrypted communications (or some particular
protocol or resource), you are an outdated and/or immoral person
whom we don't want in the IETF and who maybe doesn't deserve to
be on the Internet"... especially if someone lives in a country
or works for a company that is nervous about encrypted
communications.   The Internet I've been working to build for
well over 30 years is one that is open and flexible, not only to
connectivity from all over the world, but to allowing people to
work the way they want to rather than being victims of an "our
way or you lose" approach.

That needs to apply to the IETF.   It is entirely reasonable for
the IETF to adopt rules about how people work and interact with
others and we have done a lot of that in recent years.  But, at
least IMO, we need to remember that much of our strength and
credibility depends on diversity --not just demographic
diversity but on diversity of technical experience,
perspectives, and work habits.  Each time we say "do it this way
from now on or you might as well go away and leave the IETF to
people who work and think the way we do", we need to assume that
some people will say "ok, enough" and go away.  I can't predict
how many of them will be valuable contributors, but "we" had
best assume that some will be.   

And, for whatever it is worth --because I see this as a symptom
of a much broader set of attitudes and problems-- I think the
same principle applies to the assorted nastygrams and threats I
(and probably others) who prefer to try to think through things,
do careful analyses, and then write infrequent notes that tend
to be long as a result, get from those who seem to believe that
the only righteous way to work in the IETF involves a much
larger and more frequent number of very short messages.   I will
consider the input and try to write shorter messages when I can
but if the intent is to push everyone into expressing themselves
in one particular way or not at all, well, sooner or later I
(and maybe others) will conclude that the IETF doesn't want our
input (a couple of those messages very nearly said that) and
walk away.  That would cost the IETF a certain amount of
experience and perspective but perhaps the reality is that it is
inconvenient and unwanted.

As far as FTP access to I-Ds is concerned, I guess that I don't
care from the standpoint of my own tools and work habits,
although I do care (and am opposed to removing it) in principle.
But many of the same arguments that have been made against
keeping FTP access to those IETF materials could be made about
the RFC Editor's repository and against retaining unencrypted
access to the web-based repository.  And then I would care a
great deal (as I tried to say some weeks ago).  
 
best,
   john

[1]  The reasons are mostly unrelated to this discussion but are
due to IESG refusal to enforce naming conventions:  When naming
conventions are predictable, a decent interface to FTP that
includes what is commonly called "mget" (really a combination of
NLST, a client-end selection process, and RETR) works well and
requires a minimum number of user steps.  When they are not, a
search function like that of the datatracker gets much more
easy.  So, to a certain extent, while I trust it was not
intentional, where we are now might as well have been a sequence
of "let's make FTP access less useful so it drives people off;
then let's notice that people, having been driven off, are not
using it any more; then we notice that it is getting little use
so should discontinue it."

---

From: Roman Danyliw <[email-masked]>
Date: Tue, 24 Nov 2020 23:18:58 +0000

Hi John!

> From: John C Klensin <[email-masked]>
> Sent: Tuesday, November 24, 2020 5:05 PM
> --On Friday, November 20, 2020 11:31 +0000 Roman Danyliw <[email-masked]>
> wrote:
> 
> > Hi!
> >
> > We are about half through the feedback period and helpful discussion
> > has already occurred.  Thank you for the input.
> >
> > To help focus further discussion and to help onboard those that have
> > not weighed in yet, a summary of the thread to date can be found at
> Thank you for your note and the pointer to the summary.  I largely agree with
> Keith's comments today in https://mailarchive.ietf.org/arch/msg/ietf/ZnI-i5sB-
> d9eq7KDfkFB3Gb5ZPc
> First, it may be a trivial detail, but you undercounted those who objected. 
>  I'm
> certainly on that list unless the criterion for being counted is posting mostly-
> redundant arguments over and over again.  

Respectfully, I disagree. Parties were attributed as supporting or opposing the proposal only if they explicitly said they had a position.  If someone contributed to the conversation or posed a question, I made no assumptions of their positions.  Links to these explicit statement were provided as a short-cut into the specifics of what parties are thinking.

For example, you "+1-ed" a question on wanting details on what it costs to run the FTP service (https://mailarchive.ietf.org/arch/msg/ietf/7ImbzRpyhUmTfgcqb-hya8xU7GU/).  I made no assumptions on whether your support for wanting more details was equivalent to support or opposition of the proposal.

Based on your clarification above, I added a link for your opposition (see: https://github.com/ietf/ftp-retirement-consult-2020/commit/478c004856120501f1a1e6bdb10e28537b6574f0).  Thanks for the correction.

To everyone who has provided input, repeating the text in the summary -- "[a]s this thread consistent of many messages and interleaved ideas, if your position has been missed or misrepresented, please send corrections to the thread on [email-masked] or privately to Roman Danyliw (IESG tool representative)." 

> That, however, leads to the non-
> trivial part and one of those "everything is connected to everything else"
> problems.
> 
> I have gone back and checked through my tools and I have been using FTP
> access to I-Ds very little in the last few years [1].
> I do use FTP to access RFCs, but I do so on the RFC Editor site, not the IETF one,
> both because some of my tools are old enough that the IETF was not
> maintaining an RFC repository and because the IETF does not seem to
> understand the importance of stable locations or links.  So, in addition to not
> showing up in the summary list of those who objected, I don't show up if you
> are monitoring use of FTP access to IETF materials.  But I am opposed to making
> this change nonetheless.

Thanks for that clarification.  To be clear on scope, the disposition of the RFC Editor's services are out of scope of this proposal

> So I would like to take a step back from most of the details of the discussion
> and talk about where it seems to come from.
> Keith wrote:
> 
> I see the comments about encryption in the summary under "FTP provides a
> unique interface (Use Cases)" in the summary (and a great deal of traffic in this
> thread) as being about that same problem.
> 
> So, again, let's take a step back, noting that this gets back to my comments
> about a slippery slope between "drop FTP access" and
> access to Internet resources available and having them work well from the
> user's standpoint as well as securely?  Absolutely.  Am I in favor of educating
> users as to why encryption is good for them and they should prefer it when
> they have the option?  Yes again, although I'm not sure that is the IETF's job.
> But neither of those imply a stance that feels equivalent to "if you want to use
> unencrypted communications (or some particular protocol or resource), you are
> an outdated and/or immoral person whom we don't want in the IETF and who
> maybe doesn't deserve to be on the Internet"... especially if someone lives in a
> country or works for a company that is nervous about encrypted
> communications.   

I struggle to find text in the proposal (https://www.ietf.org/media/documents/Retiring_IETF_FTP_Service.pdf) or the usage analysis (https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJsHI4uyDlQ2dHip8jo/edit#) that suggests a position linking morality and specific technologies.

To the question of companies/users who can't user encryption, this was noted by [19] (https://mailarchive.ietf.org/arch/msg/ietf/XFJOyi9a8KsrF4vmBAghb0Zo3mo/), [72] (https://mailarchive.ietf.org/arch/msg/ietf/a0U1ofyDlbxS72MTmDGNY2YBpiI/) and [125] (https://mailarchive.ietf.org/arch/msg/ietf/Da4mZ1cC0pQ8IelsdCFjCBLXyO4/).  It was equally noted that from the data present, there don't appear to be such disadvantaged users per [25] (https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5dAb3FAgNng/), [122](https://mailarchive.ietf.org/arch/msg/ietf/b8BfvrcpLmvvjkhJ1MW8DUEzmQ8/) and  [137] (https://mailarchive.ietf.org/arch/msg/ietf/7rWVjpu1h3M3mR4OT5NqdTkSrSU/)

The Internet I've been working to build for
> well over 30 years is one that is open and flexible, not only to connectivity from
> all over the world, but to allowing people to work the way they want to rather
> than being victims of an "our way or you lose" approach.
>
> the IETF to adopt rules about how people work and interact with others and we
> have done a lot of that in recent years.  

In this case, there are published principles.  At the risk of rehashing messages already on the thread, message #25 (https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5dAb3FAgNng/) and  #69 (https://mailarchive.ietf.org/arch/msg/ietf/1lnlpXJDDl5ZSA4Xn9x4sQFPlvU/) already point to the IESG statement to maximize encrypted access and IAB issued a Statement on Confidentiality respectively as guiding principles.

> But, at least IMO, we need to
> remember that much of our strength and credibility depends on diversity --not
> just demographic diversity but on diversity of technical experience,
> perspectives, and work habits.  Each time we say "do it this way from now on
> or you might as well go away and leave the IETF to people who work and think
> the way we do", we need to assume that some people will say "ok, enough"
> and go away.  I can't predict how many of them will be valuable contributors,
> but "we" had
> And, for whatever it is worth --because I see this as a symptom of a much
> broader set of attitudes and problems-- I think the same principle applies to the
> assorted nastygrams and threats I (and probably others) who prefer to try to
> think through things, do careful analyses, and then write infrequent notes that
> tend to be long as a result, get from those who seem to believe that the only
> righteous way to work in the IETF involves a much
> consider the input and try to write shorter messages when I can but if the
> intent is to push everyone into expressing themselves in one particular way or
> not at all, well, sooner or later I (and maybe others) will conclude that the IETF
> doesn't want our input (a couple of those messages very nearly said that) and
> walk away.  That would cost the IETF a certain amount of experience and
> perspective but perhaps the reality is that it is inconvenient and unwanted.
> 
> As far as FTP access to I-Ds is concerned, I guess that I don't care from the
> standpoint of my own tools and work habits, although I do care (and am
> opposed to removing it) in principle.
> But many of the same arguments that have been made against keeping FTP
> access to those IETF materials could be made about the RFC Editor's repository
> and against retaining unencrypted access to the web-based repository.  And
> then I would care a great deal (as I tried to say some weeks ago).

To repeat again, the disposition of the RFC Editor's services are out of scope of this proposal

To the very specific philosophical point, as noted in message #92 (https://mailarchive.ietf.org/arch/msg/ietf/maUHi4gfaPAH_TfsU6XbqdYYM5Y/) and #122 (https://mailarchive.ietf.org/arch/msg/ietf/b8BfvrcpLmvvjkhJ1MW8DUEzmQ8/), rysnc still provides unencrypted access if COMSEC properties are the driver to use FTP (realizing of course the rsync and FTP are not equivalent, but most of the usage for FTP appears to be syncing file which can be satisfied by FTP as noted by https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJsHI4uyDlQ2dHip8jo/edit#)

Roman

> best,
>    john
> 
> 
> 
> [1]  The reasons are mostly unrelated to this discussion but are due to IESG
> refusal to enforce naming conventions:  When naming conventions are
> predictable, a decent interface to FTP that includes what is commonly called
> "mget" (really a combination of NLST, a client-end selection process, and RETR)
> works well and requires a minimum number of user steps.  When they are not,
> a search function like that of the datatracker gets much more easy.  So, to a
> certain extent, while I trust it was not intentional, where we are now might as
> well have been a sequence of "let's make FTP access less useful so it drives
> people off; then let's notice that people, having been driven off, are not using it
> any more; then we notice that it is getting little use so should discontinue it."

---

From: "Peter Yee" <[email-masked]>
Date: Tue, 24 Nov 2020 17:12:31 -0800

I'm in favor of turning off the FTP service. It's time has come and gone. As
someone who has worked on an FTP RFC, I appreciate the protocol's place in
history. That does not mean the IETF needs to continue to support a service
that is both redundant and little used. It should be relegated to retirement
and many fond (and not so fond) memories.

		-Peter

-----Original Message-----
From: ietf [mailto:[email-masked]] On Behalf Of Roman Danyliw
Sent: Friday, November 20, 2020 3:32 AM
To: [email-masked]
Subject: RE: Call for Community Feedback: Retiring IETF FTP Service

Hi!

We are about half through the feedback period and helpful discussion has
already occurred.  Thank you for the input.  

To help focus further discussion and to help onboard those that have not
weighed in yet, a summary of the thread to date can be found at
https://github.com/ietf/ftp-retirement-consult-2020/blob/main/status.md

Regards,
Roman

> In consultation with the Tools team (Robert, Glen, Henrik, Russ, and
Alexey),

---

From: Joseph Salowey <[email-masked]>
Date: Tue, 24 Nov 2020 17:31:07 -0800

While I too have many fond memories of FTP, I agree that it is best not to
continue to support a service that is little used where there are viable
and widely available alternatives.  The time and resources used to support
this service are better used somewhere else.

Cheers,

Joe

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 25 Nov 2020 10:03:11 -0800

John just saved me a lot of typing; let me just +1 everything he said.

For the record, I am very much opposed to shutting down the FTP service.

I have a cluster of systems using FTP to mirror I-D's.  Just the
.txt and index files.  This is trivial to perform with FTP, and
I've been running the same FTP-based mirror script for over a decade.
The cluster I use doesn't support rsync, and likely never will, yet
without that it's still more than functional enough to work as a
distributed desktop environment.  I won't declare the OS here as
that will almost certainly just devolve into a set of ad hominem
attacks against the operating environment, none of which are relevant
to the discussion.

I will also note that part of the reason ftp.ietf.org sees so little
traffic is likelty due to the highly aggressive session timeout on
the server's FTP command channel, making it pretty much unuseable
for interactive sessions. If you increased that timeout from 20
seconds to something reasonable, say five minutes, people might
start using the service interactively.

--lyndon

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Nov 2020 14:12:25 -0500

Roman,

Thank you.  Sorry for note being more clear.

A few other comments below (with a lot elided that does not seem
to be worth pursuing further right now)...

--On Tuesday, November 24, 2020 23:18 +0000 Roman Danyliw
<[email-masked]> wrote:

>...
> Respectfully, I disagree. Parties were attributed as
> supporting or opposing the proposal only if they explicitly
> said they had a position.  If someone contributed to the
> conversation or posed a question, I made no assumptions of
> their positions.  Links to these explicit statement were
> provided as a short-cut into the specifics of what parties are
> thinking.
> 
> For example, you "+1-ed" a question on wanting details on what
> it costs to run the FTP service
> (https://mailarchive.ietf.org/arch/msg/ietf/7ImbzRpyhUmTfgcqb-
> hya8xU7GU/).  I made no assumptions on whether your support
> for wanting more details was equivalent to support or
> opposition of the proposal.

Seems completely reasonable and, again, I'm sorry for not being
more clear.  So, in the interest of further clarity, my instinct
is to be opposed to this because it may lock people out or
consume their time in ways that might better be spent on IETF
technical standards work.  Whether it is worth that tradeoff or
not seems to me to depend on two things: whether dropping that
service has enough value (or saves enough in costs of various
types) to be worth it and whether the ancillary arguments hold
water.  For me, the actual cost arguments, at least so far,
don't (and that is why I asked that question). It seems to me,
as several others have suggested, that the big costs are in
setting things up initially (done decades ago) and making sure
that the scripts that keep moving copies to the right locations
aren't screwed up.  I obviously haven't looked at the systems
and scripts that do that, but many years of experience suggests
to me that, if keeping that working requires regular and
extensive effort, we have far deeper problems than whether or
not an FTP service continue to be offered.  And, fwiw, I have
far too high an opinion of the various people who have developed
and maintained the IETF's scripts and databases over the years
to consider that likely. 

>> using FTP access to I-Ds very little in the last few years
>> [1]. I do use FTP to access RFCs, but I do so on the RFC
>> Editor site, not the IETF one, both because some of my tools
>> are old enough that the IETF was not maintaining an RFC
>> repository and because the IETF does not seem to understand
>> the importance of stable locations or links.  So, in addition
>> to not showing up in the summary list of those who objected,
>> I don't show up if you are monitoring use of FTP access to
>> IETF materials.  But I am opposed to making this change
>> nonetheless.
> 
> Thanks for that clarification.  To be clear on scope, the
> disposition of the RFC Editor's services are out of scope of
> this proposal

Yes, and you said earlier that I should not be concerned about
HTTP service being cut off in favor of HTTPS-only because that
was out of scope.   I understand that now and understood it
before.  Probably I should have said so explicitly just as I
should have said explicit "I am opposed to this, in part because
I don't think the arguments for do it hold water" in the note
about costs you referred to.  But, as I suggested earlier, I've
taken what I consider to be a lot of abuse in the last several
months for posting long messages and, if I try to shorten them,
I'm going to sometimes (maybe often) make the mistake of leaving
things out.

So, to be much more explicit, I am quite aware that neither the
RFC Editor services nor a possible move to HTTPS access only are
within the scope of _this effort_.  However, whether one uses
"slippery slope" on some other vocabulary, I am also painfully
aware that, if we shut down FTP for I-Ds based on arguments that
not enough relevant people are using it, that no morally
acceptable person should be using it, or that it violates the
IETF's norms about security and encryption, then it is a tiny
step to some other effort arguing that the RFC Editor shouldn't
be supporting it, using both the (unrefuted) arguments in part
of this discussion and the evidence that FTP support was
discontinued and no one died.  And it would be an equally tiny
step to say "HTTP access violates our principles so it should be
shut down and we got rid of cleartext access with FTP and
discontinuing HTTP should be ok too."   I am _not_ suggesting
that anyone has an explicit, multi-stage plan for either, only
that these things happen and that, if those discussions come up
a year or five from now, it is important to have the reasons why
we are doing this absolutely clear and to not open those doors
and leave them open.

>...
> I struggle to find text in the proposal
> (https://www.ietf.org/media/documents/Retiring_IETF_FTP_Servic
> e.pdf) or the usage analysis
> (https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0Vs
> JsHI4uyDlQ2dHip8jo/edit#) that suggests a position linking
> morality and specific technologies.

See above.  I was not referring to anything you had written,
only to some of the comments in the discussion that appeared to
me rather strongly suggest that no one should be using FTP
anyway because it is insecure and violates IETF
policies/preferences in that area, that it is an unfortunate
artifact of the time before we discovered The One True way with
the web and should be abolished, etc.  If my referring to those
as moral arguments is an inappropriate exaggeration, I
apologize, but I had hoped people would get the point.

> To the question of companies/users who can't user encryption,
> this was noted by [19] [72] and [125]
>...
>  It was equally noted that from the data present,
> there don't appear to be such disadvantaged users per [25]
> [122] and  [137]

And, here, again, I'm trying to look ahead.  Maybe we are going
to dodge the bullet in the US, but there are clear international
trends toward restrictions on encryption and/or requirements for
law enforcement access, keys deposited with governments, etc.
We have deliberately designed many of our protocols to make the
latter hard (and I personally think that was exactly right).
But suppose some country (to make this a concrete example, say
France because they have done it before) bans encryption without
explicit government permission/ license).  Unless we are going
to decide that we are not interested in input from, or access to
files by, anyone living in France, we end up in a mad scramble,
one that might be used to discredit everything we have done to
push for more encryption and that gets us back to where we are
today.  Instead, I think we are better off if we take the
position that these document are free and freely available, and
that while, in line with our announced policies, we are making
encrypted access available and recommend people use it, we
consider it important to be sure those materials who are
available to everyone, even in restricted regimes.  YMMD.
Because that situation has not arisen, yet and in the last
decade or so, in France or elsewhere that we know about, the
usage statistics are not going to show it.

Similarly, I'm not a security expert or watching security
issues, but I'm noticed that several enterprises and at least
one vendor of enterprise firewalls/ security appliances have
concluded that the solution to malware-over-HTTPS is to lie to
servers about keys, get everything into cleartext on the
firewall, and then pass it through.  Depending on one's threat
model (which the IETF recommendations have not, IMO, been
explicit about), the protection being offered may be marginal to
zero and, perhaps, everyone would be better off if the public
information were retrieved and transmitted in the clear.
However, even if we are fine with that situation, any
measurements we make show users within those enterprises
accessing the IETF servers via HTTPS.

All of the above discussion about security and encryption should
be irrelevant.  Considering that IETF is already not offering a
full and conforming FTP service, if the only thing at issue here
is FTP and there is a plausible long-term guarantee of HTTP
access as well as HTTPS, I don't see tremendous harm in dropping
it.  However, I am still concerned enough about the IETF, by its
actions, endorsing the notion that, at the applications layer,
everyone should be forced to do things that same way that I
would prefer to be listed as opposed.
 
>> well over 30 years is one that is open and flexible, not only
>> to connectivity from all over the world, but to allowing
>> people to work the way they want to rather than being victims
>> of an "our way or you lose" approach.
>> 
>> That needs to apply to the IETF.   It is entirely reasonable
>> for the IETF to adopt rules about how people work and
>> interact with others and we have done a lot of that in recent
>> years.  
> 
> In this case, there are published principles.  At the risk of
> rehashing messages already on the thread, message #25
> (https://mailarchive.ietf.org/arch/msg/ietf/py_9b486x8x2io6d5d
> Ab3FAgNng/) and  #69
> (https://mailarchive.ietf.org/arch/msg/ietf/1lnlpXJDDl5ZSA4Xn9
> x4sQFPlvU/) already point to the IESG statement to maximize
> encrypted access and IAB issued a Statement on Confidentiality
> respectively as guiding principles.

And, at the risk of repeating myself, I read the IAB and IESG
statements as encouraging greater availability and user of
encryption on the network.  I don't see anything in either of
them that can reasonably be interpreted as "if you don't agree
with us, or if circumstances prevent your using encryption, than
you should get off our network or at least not try to
participate in the IETF".

> are out of scope of this proposal

Understood but see above.  I, again, would argue that this would
set a precedent that could easily be misused (and probably would
be) in a way that would impact those services.

> 
> To the very specific philosophical point, as noted in message
> #92
> (https://mailarchive.ietf.org/arch/msg/ietf/maUHi4gfaPAH_TfsU6
> XbqdYYM5Y/) and #122
> (https://mailarchive.ietf.org/arch/msg/ietf/b8BfvrcpLmvvjkhJ1M
> W8DUEzmQ8/), rysnc still provides unencrypted access if COMSEC
> properties are the driver to use FTP (realizing of course the
> rsync and FTP are not equivalent, but most of the usage for
> FTP appears to be syncing file which can be satisfied by FTP
> as noted by
> https://docs.google.com/document/d/1JAXspeaMWFl8ML3hSezFSM0VsJ
> sHI4uyDlQ2dHip8jo/edit#)

First of all, to the extent to which the issue is "don't make
IETF participants change their tools because the time would be
better spent on IETF technical work", "you could replace FTP
with foo" (whether 'foo' is rsync or something else) is not
really helpful.  If it is being used mostly for synchronization,
then it would be an interesting (and very inexpensive)
experiment to try to call the attention of those users to the
fact (or at least our belief) that rsync would probably work
better for them and then see if that produces any change in
behavior.  But, again, from my point of view, our goal should be
encouraging use of IETF resources and participation in the IETF,
not seeing what barriers we can erect because would-be
participants could be doing things in other ways.  And, fwiw,
when I was still using FTP regularly for retrieving I-Ds, it was
not to sync them, it was because the underlying FTP support was
present in my favorite text editor, making it very easy to
construct keyboard macros that incorporated the relevant
pathnames and skeleton.  Doing that in the editor is not just
because I can but because that is where I typically want the
file.  Because there is not the same level of support for rsync,
doing the same thing would require me to write code in ELL or a
peculiar dialect of LISP rather than constructing a keyboard
macro, a considerably larger (although obviously still feasible)
task.  

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Nov 2020 15:09:47 -0500

--On Wednesday, November 25, 2020 10:03 -0800 Lyndon Nerenberg
<[email-masked]> wrote:

> I will also note that part of the reason ftp.ietf.org sees so
> little traffic is likelty due to the highly aggressive session
> timeout on the server's FTP command channel, making it pretty
> much unuseable for interactive sessions. If you increased that
> timeout from 20 seconds to something reasonable, say five
> minutes, people might start using the service interactively.

I had forgotten that particular annoyance, but that was another
reason I stopped using FTP interactively (to retrieve several
I-Ds in sequence) rather than the editor macros, which still
work fine for single I-D retrievals if am can sure I know what
the name, including version number, is.   I don't know what
Lyndon is doing but, while five minutes seems a tad long, 20
seconds is too short to retrieve something, think for a bit, and
then issue another command... and probably too short for clients
in many environments to perform "mget" by sending NLST, getting
back a list of names, builting a mini-script, and then start
sending RETR commands.

   john

---

From: "John Levine" <[email-masked]>
Date: 25 Nov 2020 15:54:16 -0500

In article <[email-masked]> you write:
>distributed desktop environment. ...

I don't care what OS you're running, but I do observe that rsync has
been widely available for well over 20 years. Anything too old to run
rsync strikes me as a corner case of a corner case.

R's,
John

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 25 Nov 2020 21:43:57 +0000

Lyndon,

your email brings up the central issue from the other side:

> The cluster I use doesn't support rsync, and likely never will

the issue at stake is technical debt, and specifically who pays for it: 
ietf llc, or ietf stakeholders.

Your argument is the same as that of several other people, i.e. there's 
technical debt on your side and your preference would be for someone 
else to shoulder some or all of this.

This is not a criticism, btw, and it's something we can all relate to: 
everyone has technical debt.  It doesn't matter whether it's a large 
enterprise - where technical debt might form part of the financials - or 
if it's granny, whose desktop is still running XP, and where all the 
maintenance is handled by her grandchild - payment in cookies.  The only 
question we struggle with is when it's reasonable for us to shoulder the 
costs, and when we can expect others to.

Nick

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 25 Nov 2020 13:57:00 -0800

> rsync strikes me as a corner case of a corner case.

As I said, ad hominem arguments.

--lyndon

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 25 Nov 2020 14:07:47 -0800

Are you equating "using ancient technology" with "accumulating technical debt"?

<insert condescending joke how AI, an ancient technology proposed by Turing back in 1950>

On 11/25/20 1:43 PM, Nick Hilliard wrote:
> the issue at stake is technical debt, and specifically who pays for it: ietf llc, or ietf stakeholders.
> 
> Your argument is the same as that of several other people, i.e. there's technical debt on your side and your preference would be for someone else to shoulder some or all of this.
> 
> This is not a criticism, btw, and it's something we can all relate to: everyone has technical debt.  It doesn't matter whether it's a large enterprise - where technical debt might form part of the financials - or if it's granny, whose desktop is still running XP, and where all the maintenance is handled by her grandchild - payment in cookies.  The only question we struggle with is when it's reasonable for us to shoulder the costs, and when we can expect others to.
> 
> Nick

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 25 Nov 2020 22:09:37 +0000

> Are you equating "using ancient technology" with "accumulating technical 

lemme answer that question as soon as this upgrade to big sur is 
finished - your understanding is appreciated.

Nick

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 25 Nov 2020 14:23:40 -0800

Are you equating "planned obsolescence in proprietary technology designed to separate people from their money" with "ancient technology that, like good wine, get better with time"?

On 11/25/20 2:09 PM, Nick Hilliard wrote:
> lemme answer that question as soon as this upgrade to big sur is finished - your understanding is appreciated.
> 
> Nick

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 25 Nov 2020 22:43:05 +0000

Marc Petit-Huguenin wrote on 25/11/2020 22:23:
> Are you equating "planned obsolescence in proprietary technology 
> designed to separate people from their money" with "ancient technology 
> that, like good wine, get better with time"?

no, simply observing that we all spend an extraordinary amount of time 
and effort handling technical debt of one form or another, which is 
maybe one of the reasons that we all slightly resent it when the 
suggestion is posed to us by third parties.

Nick

---

From: Keith Moore <[email-masked]>
Date: Wed, 25 Nov 2020 17:47:34 -0500

> rsync strikes me as a corner case of a corner case.

I don't get this argument.   You have a tool that you find useful, 
great!   That's not an excuse to disrupt use of a tool that other people 
find useful.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 25 Nov 2020 17:50:50 -0500

> it: ietf llc, or ietf stakeholders. 

That's an _extremely_ biased (and I would argue insulting) way of 
framing the issue.  FTP is not technical debt, it's a valuable feature 
that not everyone happens to utilize (maybe because that's not their 
preference, maybe because they've never seen the advantages).

Or if you want to put it that way, HTTP/HTML/CSS/JS/etc. has a lot of 
technical debt too: a tremendous amount of code overhead needed to use 
it effectively, really bloated clients, dysfunctional user interface 
when used outside of a very narrow range, high maintenance costs, etc.

---

From: "John Levine" <[email-masked]>
Date: 25 Nov 2020 21:17:47 -0500

>great!   That's not an excuse to disrupt use of a tool that other people 

To point out the obvious, the IETF's staff and financial resources are
not infinite. Based on Roman's numbers, we seem to be talking about a
dozen FTP users and at least thousands if not tens of thousands of
rsync and https users. I find arguments that we should ignore the
numbers because a few individuals think they might be inconvenienced
unpersuasive.

The previous message about technical debt summed up the issue pretty well.

---

From: Keith Moore <[email-masked]>
Date: Wed, 25 Nov 2020 21:21:08 -0500

On 11/25/20 9:17 PM, John Levine wrote:

> rsync and https users.

As I've said earlier, I think those numbers are being misinterpreted.

Also, it's now obvious that the IETF FTP server is poorly configured 
since it's timing out the control connection after 20 seconds.   That's 
simply unreasonable.   If usage is down because the server is 
misconfigured, that's hardly surprising, but it doesn't mean the server 
wouldn't be used more if it were fixed.

>   I find arguments that we should ignore the

No, that's just another example of ageist prejudice.

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 03:09:57 +0000

Hi Lyndon!

> From: ietf <[email-masked]> On Behalf Of Lyndon Nerenberg
> Sent: Wednesday, November 25, 2020 1:03 PM
> To: [email-masked]

[snip]

> I have a cluster of systems using FTP to mirror I-D's.  Just the .txt and index
> files.  This is trivial to perform with FTP, and I've been running the same FTP-
> based mirror script for over a decade.
> The cluster I use doesn't support rsync, and likely never will, yet without that
> it's still more than functional enough to work as a distributed desktop
> environment.

To have as a backup, there is also ftp://ftp.rfc-editor.org/in-notes/internet-drafts against which I-Ds can be mirrored.

Roman

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Nov 2020 23:30:53 -0500

--On Thursday, November 26, 2020 03:09 +0000 Roman Danyliw
<[email-masked]> wrote:

>...
> To have as a backup, there is also
> ftp://ftp.rfc-editor.org/in-notes/internet-drafts against
> which I-Ds can be mirrored.

Roman,

I didn't know about that and, at least for me, that completely
changes the equation.  If there is at least one IETF (or IETF
LLC)-supported site that is kept synchronized with the IETF I-D
collection and offers FTP, I don't see any strong reason why
there needs to be access to the IETF repository on the IETF
site.  Even for those who need to change scripts, changing one
site and path for another (either in human memory or in a
script) should not be a big deal and, IIR, IETF has moved things
around often enough that most FTP users have had to do that once
or twice already.

On the other hand, if the LLC has to support, or fund AMS to
support, FTP access to a repository on the RFC Editor site/path,
it seems that the case that there are significant marginal costs
for maintaining FTP access to the IETF repository just got a
little more dubious.

thanks,
   john

---

From: Keith Moore <[email-masked]>
Date: Wed, 25 Nov 2020 23:54:48 -0500

On 11/25/20 11:30 PM, John C Klensin wrote:

> If there is at least one IETF (or IETF

I'm probably ok with that also... as long as _that_ ftp server doesn't 
go away anytime soon or without a good reason.   To me it's more 
important that the service exist than that it be run by IETF.  
(especially since the IETF server is being run so poorly)

Maybe just change the DNS for ftp.ietf.org to a CNAME that points to it?

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 Nov 2020 08:25:01 +0100

On 2020-11-25, at 23:50, Keith Moore <[email-masked]> wrote:
> 
> FTP is not technical debt

I’m pretty sure the consensus opinion among system administrators is that it very, very much is (if they even have heard about FTP).

Our department stopped offering FTP access in January 2008.
The situation with FTP server implementations had deteriorated, and after the crash of a (then already obsolete) storage array that carried the data nobody cared enough to have it set up again.
(The university computing center still has one; the newest files I can find there are from 2012.  Ah the kites archive mirror...
However, there also is a sea-ice archive that offers its considerable collection of geoscience data via ftp.)

I personally last used FTP for production with a photo printer service, which actually offered FTP upload as an efficient alternative to click-per-picture upload.
Of course, browsers now have great support for uploading collections of files, so that finally went away half a decade ago (probably as part of the general security ramp-up in reaction to the Snowden revelations, but I don’t have detailed notes of how I uploaded pictures there when).

For my personal use of IETF data, I simply rsync everything I need to my laptop and take it from there.  All the advantages that you note for having them in a filesystem, and zero wait.  Works offline, too.  A little over 10 gig or so, including the mail archive (part of which I also sync via imap, so I’m not always up to date on that).

Security considerations, NAT/firewall issues, and the wide usage of cloud storage (dropbox, google drive etc.) have made FTP-based services obsolete.
Of course, the protocol still exists, so consenting adults can use it.
The question was whether the IETF operationally needs to continue to consent, and it looks pretty grim for that from this vantage point.

I’m a little reminded of the brouhaha in 1998 when Steve Jobs took away the floppy drives from the iMac.  Yes, floppies had been useful.  For a while.  Being able to decide to stop using something, when its time has come, is a strength.

Grüße, Carsten

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 26 Nov 2020 16:59:48 +0900

Nick Hilliard wrote:

> and effort

Wasting ML bandwidth.

As there already are substantial amount of counter argument, we
should just shutdown this thread and continue offering ftp service,
which is the most time and effort saving operation.

						Masataka Ohta

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 11:23:24 +0000

Hi Keith!

> Sent: Wednesday, November 25, 2020 11:55 PM
> To: [email-masked]
> > If there is at least one IETF (or IETF LLC)-supported site that is
> > kept synchronized with the IETF I-D collection and offers FTP, I don't
> > see any strong reason why there needs to be access to the IETF
> > repository on the IETF site.  Even for those who need to change
> > scripts, changing one site and path for another (either in human
> > memory or in a
> > around often enough that most FTP users have had to do that once or
> > twice already.
> 
> I'm probably ok with that also... as long as _that_ ftp server doesn't go away
> anytime soon or without a good reason.   To me it's more important that the
> service exist than that it be run by IETF. (especially since the IETF server is being
> run so poorly)

"Run so poorly" is a strong statement.  What is the basis of your value judgement?  What isn't working for you?  Have you reported it so it can be improved?

If you are specifically referencing the timeout first mentioned in https://mailarchive.ietf.org/arch/msg/ietf/uaCQP0VPcLWzDe7hRkLiFAx4MLU/ (FTP timeout after 20 seconds), I consulted with the Tools team who tells me this configuration has been in place for over a decade.  In my quick check on ietf@ and their checking in issue reporting in recent years, this is the first time this issue has been raised.

If you have a specific issue, please report it to ietf-action@, check in with ietf@ to see how it is pervasive, or tell the iesg@.  

Roman

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 11:37:23 +0000

Hi Lyndon!

> environment.  I won't declare the OS here as that will almost certainly just
> devolve into a set of ad hominem attacks against the operating environment,
> none of which are relevant to the discussion.

Understood.

> I will also note that part of the reason ftp.ietf.org sees so little traffic is likelty
> due to the highly aggressive session timeout on the server's FTP command
> channel, making it pretty much unuseable for interactive sessions. If you
> increased that timeout from 20 seconds to something reasonable, say five

This seems to reference another use cases -- interactive access.  Is that also from the same end point identified above?  If so, you've asked us not to discuss it, so ignore my ask.  However, if this is a different client, can you share more details on your client. I ask because it is my understanding that most FTP clients now auto-reconnect hiding any such issues.  Responding privately is fine too.

As to linking this configuration to low FTP usage, I'm not so sure.  As I noted in https://mailarchive.ietf.org/arch/msg/ietf/l2rQFow9G-Nc1U04dLTCzsv5ka0/, this configuration has been in place for at least a decade and no user feedback has been voiced to indicate this is a problem (until now).

Regards,
Roman

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 07:27:05 -0500

> I’m pretty sure the consensus opinion among system administrators is that it very, very much is (if they even have heard about FTP).
Well, this is the usual problem with system administrators. They're not 
the ones who actually use and need the services that they provide, and 
because of that distance, they don't always properly assess the value of 
those services.
> However, there also is a sea-ice archive that offers its considerable collection of geoscience data via ftp.)
Every server is different and should be evaluated separately.
> I personally last used FTP for production with a photo printer service, which actually offered FTP upload as an efficient alternative to click-per-picture upload.

I personally use FTP every week or so because of features mentioned in a 
recent message of mine, though only for "anonymous" downloads.  The same 
FTP client that lets me easily move files to or from my phone also 
supports sshfs, so I use that whenever authentication is needed.    But 
that client would be less useful if it didn't support FTP, because I 
don't have the ability nor inclination to install an ssh server 
everywhere that I need to access.

Web browsers are overall very cumbersome tools for many purposes, so I'm 
glad there are alternatives to them.

> For my personal use of IETF data, I simply rsync everything I need to my laptop and take it from there.  All the advantages that you note for having them in a filesystem, and zero wait.  Works offl…
Glad it works for you.   What I don't accept is the implication that 
because someone _can_ use rsync to sync the entire collection, that's 
what everyone who occasionally needs access to the collection _should_ 
do in every circumstance.
> Security considerations, NAT/firewall issues, and the wide usage of cloud storage (dropbox, google drive etc.) have made FTPbased services obsolete.
Everyone who is still using it (which appears to be a significant 
fraction of active IETF participants) might not agree with you.
> I’m a little reminded of the brouhaha in 1998 when Steve Jobs took away the floppy drives from the iMac.  Yes, floppies had been useful.  For a while.  Being able to decide to stop using something,…

I'm actually pretty fed up with Apple because of its constant changing 
of interfaces that work.   Apple's newer iPhones no longer serve the 
purposes that drove my purchase of my original iPhone, and they've also 
become hideously expensive and less usable and less functional at the 
same time.   So yeah, if you're a fan of gratuitous incompatibility, by 
all means buy Apple products, but don't expect that this is the Right 
Thing for others to do.

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 14:19:30 +0000

Hi John!

> Sent: Wednesday, November 25, 2020 11:31 PM
> To: Roman Danyliw <[email-masked]>; Lyndon Nerenberg <[email-masked]>
> Cc: [email-masked]
> > ftp://ftp.rfc-editor.org/in-notes/internet-drafts against  which I-Ds
> >can be mirrored.
> 
> Roman,
> 
> I didn't know about that and, at least for me, that completely changes the
> equation.  If there is at least one IETF (or IETF LLC)-supported site that is kept
> synchronized with the IETF I-D collection and offers FTP, I don't see any strong
> reason why there needs to be access to the IETF repository on the IETF site.
> Even for those who need to change scripts, changing one site and path for
> another (either in human memory or in a
> script) should not be a big deal and, IIR, IETF has moved things around often
> enough that most FTP users have had to do that once or twice already.

Since we talked exchanged notes on being clear on positions (see: https://mailarchive.ietf.org/arch/msg/ietf/GXzI5CRCVLhfKzFR4s5BwKmjhio/), am I correct in interpreting that you have moved from "opposed" to "no objection contingent on continued operations of an alternative ftp source such as ftp://ftp.rfc-editor.org"?

> On the other hand, if the LLC has to support, or fund AMS to support, FTP
> access to a repository on the RFC Editor site/path, it seems that the case that
> there are significant marginal costs for maintaining FTP access to the IETF
> repository just got a little more dubious.

I'm speaking for the IESG, so I (we) have no purview over contract.  With that caveat and being completely on the outside, I will repeat what was mentioned earlier that operational complexity get reduced with fewer services to run.  It also reduced attack surface (perhaps one was a little behind in updates or had a misconfiguration).

Roman

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 09:29:01 -0500

On 11/26/20 6:23 AM, Roman Danyliw wrote:

>   What isn't working for you?  Have you reported it so it can be improved?
>
> If you are specifically referencing the timeout first mentioned inhttps://mailarchive.ietf.org/arch/msg/ietf/uaCQP0VPcLWzDe7hRkLiFAx4MLU/  (FTP timeout after 20 seconds), I consulted with the Tools team who tells me this configuration has been in place for over a decade.  In my quick check on ietf@ and their checking in issue reporting in recent years, this is the first time this issue has been raised.

I noticed many years ago that the ietf ftp server was "fussy", and 
noticed this again after testing the server in the context of this 
conversation.  I just never had time to figure out why.

But knowing that the control connection times out after 20 seconds, it's 
pretty obvious that that's going to be annoying for interactive users 
(there are lots of interruptions that take more than 20 seconds).

It should also be obvious that it's going to probably break some 
non-interactive programs.  (think about it, if a program is transferring 
files one-at-a-time, and any file takes longer than 20 seconds to 
transfer, the program is going to have to either explicitly recover from 
the closing of the control connection, or simply fail).

What I saw the other day when trying to do "grep 'judgment' 
/rfc/rfc*.txt" was a lot of "no such file or directory" errors that 
should not have happened because the files were actually there.   I 
don't know if this was because of the control connection timeout, but 
it's certainly plausible.

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 14:39:25 +0000

Hi Keith!

> service exist than that it be run by IETF. (especially since the IETF server is being

As I just noted to John (https://mailarchive.ietf.org/arch/msg/ietf/FDhwYSxK0WSV0cdwf2aZiPfMQbI/), I want to be crystal clear on positions -- have you moved from "opposed" to "no objection contingent on continued operations of an alternative ftp source such as ftp://ftp.rfc-editor.org"?

Unlike John, we had discussed a broad array of content, so I also want to point out that the ftp.rfc-editor.org site has different content that ftp.ietf.org -- RFCs and I-Ds are there.  However, for example, the mailing list archives and charters are NOT on the rfc-editor site.

> Maybe just change the DNS for ftp.ietf.org to a CNAME that points to it?

When considering what do if the ftp.ietf.org were to be deprecated, the proposal suggested as few things related to handling document pointers and explicitly turning it off (i.e., no service on that port or resolution of that name).  A few parties (https://mailarchive.ietf.org/arch/msg/ietf/XFJOyi9a8KsrF4vmBAghb0Zo3mo/, https://mailarchive.ietf.org/arch/msg/ietf/SZBuNoGOiSajpsvurZ9hQuiNINc/, https://mailarchive.ietf.org/arch/msg/ietf/-nZbFI-D1bp-o-LYGJNA_U1k9fQ/) suggested something less dramatic by putting in a pointer-like README file as the sole content.  Is your position that you would preferred to instead CNAME ftp.ietf.org to ftp.rfc-editor.org instead?

I ask because the directory layout from ftp.ietf.org and ftp-rfc.editor.org is NOT the same.  My question is which seems better, when you go to ftp.iett.org and the directory you expect is not there, but you find a README on where to go; OR you are redirected to another server which has the content, but no hint on where to go (although it might be clear from the presented directories)

Regards,
Roman

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 09:45:14 -0500

On 11/26/20 9:19 AM, Roman Danyliw wrote:

> Since we talked exchanged notes on being clear on positions (see:https://mailarchive.ietf.org/arch/msg/ietf/GXzI5CRCVLhfKzFR4s5BwKmjhio/), am I correct in interpreting that you have moved from "opposed" to "no objection contingent on continued operations of an alternative ftp source such asftp://ftp.rfc-editor.org"?

You didn't ask me, but that's my position.  I won't insist that IETF 
host the service, but I do think it needs to continue to exist, at least 
until there's been time to put up and test and transition to something 
that provides similar services but works better (perhaps webdav).

My fallback position was to set up such a service myself, but even for 
such a simple service that's a big commitment if the service is going to 
be useful in the long-term, and I'm not already in the business of 
running public-facing servers. (OTOH, if AWS's FTP hosting supported 
public access, I would have already set up such a service.)   But if the 
rfc editor is going to continue to maintain their FTP server, that's 
sufficient, particularly when it is now clear that IETF management would 
really resent maintaining it.

More broadly, this conversation has convinced me that IETF cannot be 
relied on to provide the tools that IETF participants need, and in fact 
that IETF management has become hostile to participation in IETF by some 
kinds of people.   So if IETF is going to continue to be a viable 
consensus-based standards-making organization, either the management 
needs to do a significant about-face in its attitude toward 
participation, or IETF participants need to take on the burden of 
providing those tools.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 09:48:58 -0500

On 11/26/20 9:39 AM, Roman Danyliw wrote:

> When considering what do if theftp.ietf.org  were to be deprecated, the proposal suggested as few things related to handling document pointers and explicitly turning it off (i.e., no service on that port or resolution of that name).  A few parties (https://mailarchive.ietf.org/arch/msg/ietf/XFJOyi9a8KsrF4vmBAghb0Zo3mo/,https://mailarchive.ietf.org/arch/msg/ietf/SZBuNoGOiSajpsvurZ9hQuiNINc/,https://mailarchive.ietf.org/arch/msg/ietf/-nZbFI-D1bp-o-LYGJNA_U1k9fQ/) suggested something less dramatic by putting in a pointer-like README file as the sole content.  Is your position that you would preferred to instead CNAMEftp.ietf.org  toftp.rfc-editor.org  instead?
>
> I ask because the directory layout fromftp.ietf.org  and ftp-rfc.editor.org is NOT the same.  My question is which seems better, when you go toftp.iett.org  and the directory you expect is not there, but you find a README on where to go; OR you are redirected to another server which has the content, but no hint on where to go (although it might be clear from the presented directories)

Ok, fair.  Unfortunately FTP doesn't support a redirect, so CNAME is the 
best alternative, and FTP servers have been dealing with CNAMEs for 
decades.   But as you point out, it doesn't work well if the two servers 
don't have pretty much the same content and organization.  Next best 
alternative is probably to run a vestigal ftp server and put something 
in the welcome message, but then you're still running an FTP server and 
may as well just keep serving content too.

Keith

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 26 Nov 2020 14:52:33 +0000

Hey Keith

> Sent: Thursday, November 26, 2020 9:45 AM
> To: [email-masked]
> > Since we talked exchanged notes on being clear on positions
> (see:https://mailarchive.ietf.org/arch/msg/ietf/GXzI5CRCVLhfKzFR4s5BwKmjhio
> /), am I correct in interpreting that you have moved from "opposed" to "no
> objection contingent on continued operations of an alternative ftp source such
> asftp://ftp.rfc-editor.org"?
> 
> You didn't ask me, but that's my position. 

Ooops.  Cut-and-paste error from my earlier note to John (https://mailarchive.ietf.org/arch/msg/ietf/FDhwYSxK0WSV0cdwf2aZiPfMQbI/) :-).  Sorry.

Roman

 I won't insist that IETF host the
> service, but I do think it needs to continue to exist, at least until there's been
> time to put up and test and transition to something that provides similar
> services but works better (perhaps webdav).
> 
> My fallback position was to set up such a service myself, but even for such a
> simple service that's a big commitment if the service is going to be useful in the
> long-term, and I'm not already in the business of running public-facing servers.
> (OTOH, if AWS's FTP hosting supported public access, I would have already set
> up such a service.)   But if the rfc editor is going to continue to maintain their
> FTP server, that's sufficient, particularly when it is now clear that IETF
> management would really resent maintaining it.
> 
> More broadly, this conversation has convinced me that IETF cannot be relied on
> to provide the tools that IETF participants need, and in fact that IETF
> management has become hostile to participation in IETF by some kinds of
> people.   So if IETF is going to continue to be a viable consensus-based
> standards-making organization, either the management needs to do a
> significant about-face in its attitude toward participation, or IETF participants
> need to take on the burden of providing those tools.
> 
> Keith
>

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Nov 2020 11:15:57 -0500

Roman,

Thanks.  Inline below.

>> I didn't know about that and, at least for me, that
>> completely changes the equation.  If there is at least one
>> IETF (or IETF LLC)-supported site that is kept synchronized
>> with the IETF I-D collection and offers FTP, I don't see any
>> strong reason why there needs to be access to the IETF
>> repository on the IETF site. Even for those who need to
>> change scripts, changing one site and path for another
>> (either in human memory or in a
>> script) should not be a big deal and, IIR, IETF has moved
>> things around often enough that most FTP users have had to do
>> that once or twice already.
 
> (see:
> https://mailarchive.ietf.org/arch/msg/ietf/GXzI5CRCVLhfKzFR4s5
> BwKmjhio/), am I correct in interpreting that you have moved
> from "opposed" to "no objection contingent on continued
> operations of an alternative ftp source such as
> ftp://ftp.rfc-editor.org"?

"only weak objections contingent..." would be a tad better, but
let me comment on a few things below and then explain the
difference.

>> support, FTP access to a repository on the RFC Editor
>> site/path, it seems that the case that there are significant
>> marginal costs for maintaining FTP access to the IETF
> I'm speaking for the IESG, so I (we) have no purview over
> contract.  With that caveat and being completely on the
> outside, I will repeat what was mentioned earlier that
> operational complexity get reduced with fewer services to run.
> It also reduced attack surface (perhaps one was a little
> behind in updates or had a misconfiguration).

Understood.  However, whatever niceties we might have about the
boundaries among decisions made by the IESG, decisions made by
whomever or whatever makes decisions for the RFC Editor
Function, and decisions/contracts managed by the IETF LLC, those
distinctions are fairly invisible from a distance.  That is
important for two reasons.  First, if this is really just an
operational matter, even one of operational complexity, then it
should not be the IESG's problem at all -- the LLC should be
discussing it as an operational matter and consulting the
community.   No, I don't think starting over would be at all
helpful but if you (and presumably the rest of the IESG are
"completely on the outside", then I'm not sure why you
(severally) have started this discussion.   Second, your
statement takes us into the uncomfortable territory of plausible
deniability.  Suppose the IETF drops the FTP service because
almost no one is using it (an argument made several times in
this discussion) and because it adds operational complexity and
increases security risks.  Then (especially in the absence of an
RSE or successor function with authority) the LLC wakes up some
morning in the near future and says "The IETF has concluded,
after an extensive community discussion, that an FTP service is
unnecessary, unused, adds operational complexity, and increases
risks.  Certainly the same arguments apply to the RFC Editor's
FTP service(s), so let's tell AMS to shut it down."   Absolutely
nothing would prevent them from doing that and, if we are really
treating these things as complete separate entities which are
"completely on the outside" with respect to each other, then "no
objection contingent..." turns back into a strong objection
unless whoever or whatever makes the decision for the IETF
repository (I'm presuming the IESG is taking on that authority)
makes a firm commitment that, if the RFC Editor FTP
functionality is shut down and not replaced by some other FTP
availability of I-Ds (and RFCs if they are involved) under the
general IETF-RFC Editor-IETF LLC-ISOC umbrella, the IETF service
will be immediately turned back on.

Put into a sentence you can put in the summary, I have no
objection to the IETF dropping its FTP service if there is a
firm commitment that those documents and and resources will
continue to be available from some site within the general IETF
family.  If that assurance cannot be given, then I am back onto
the strong objection list.

In addition, I don't know if this is worth belaboring, but a
number of the arguments that have been made, particularly those
that imply that FTP service should be shut down because people
who want to use it are, e.g., just horribly backward and/or
should join the current century, seem to be to be spurious
and/or insulting.  I would hope they would not be included in
any future summary  of why discontinuing FTP service or, better,
explicitly disclaimed.  For example and because of my concerns
about actions others (including the LLC and/or RFC Editor
Function) might decide to take based on an IETF/IESG decision,
it seems to me to be very important that "our" reasons for
whatever decisions we made be very clear and that the implied
insults of those arguments should have no part of them.

    best,
     john

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 11:25:11 -0500

On 11/26/20 11:15 AM, John C Klensin wrote:

> I have no
+1 for myself.
> and/or insulting.
+1.   And also ageist.
>   I would hope they would not be included in

IMO the insults weaken the (already weak and poorly supported) arguments 
for deprecating FTP, because they reveal a very evident prejudice on the 
part of those making the insults.

It should also be clear by now that IESG is not well-placed to decide 
which tools people should be using.  This is not their job,  not what 
they're chosen for, and a distraction from their main roles.

Keith

---

From: Dave Cridland <[email-masked]>
Date: Thu, 26 Nov 2020 16:26:24 +0000

On Wed, 25 Nov 2020 at 18:03, Lyndon Nerenberg <[email-masked]> wrote:

> The cluster I use doesn't support rsync
>

I appreciate this is slightly off-topic, but given the consistent
suggestion that "rsync" should replace FTP, perhaps this is the best place
to ask - I can find an implementation of rsync easily enough, but I can't
find the specification published as an open standard anywhere - do you know
where I can find it?

Thanks,

Dave.

---

From: "John Levine" <[email-masked]>
Date: 26 Nov 2020 12:08:26 -0500

>find the specification published as an open standard anywhere - do you know

It was originally described in this 1996 tech report from the Australian National University:

https://www.andrew.cmu.edu/course/15-749/READINGS/required/cas/tridgell96.pdf

The design and motivation are described in more detail in Andrew Tridgell's 1999 PhD Thesis:

https://openresearch-repository.anu.edu.au/bitstream/1885/144682/2/b20302253_Tridgell_A.pdf

If you're wondering if there's an RFC for it, nope. RFC 5781 specifies
the rsync: URI but refers back to http://rsync.samba.org/ for details
on rsync.

I don't suppose anyone would object if we made an RFC out of the spec
but given that the tech report, thesis, and multiple implementations
are widely available without restriction (other than GPL on some of
the code) it doesn't seem worth a lot of effort.

R's,
John

---

From: Job Snijders <[email-masked]>
Date: Thu, 26 Nov 2020 17:20:43 +0000

> find the specification published as an open standard anywhere - do you know

Fun question!

The RSYNC algorithm is outlined in these (excellent) papers by Andrew
Tridgell: https://rsync.samba.org/~tridge/phd_thesis.pdf and
https://www.andrew.cmu.edu/course/15-749/READINGS/required/cas/tridgell96.pdf

Currently only two interoperable implementations of the RSYNC protocol exist:

GPL rsync, aka 'the original'
    source: https://github.com/WayneD/rsync
    web: https://rsync.samba.org/
    initial release: 1996

openrsync, a clean-room implementation by the OpenBSD project
    source: https://cvsweb.openbsd.org/src/usr.bin/rsync/
    web: https://www.openrsync.org/
    initial release: 2019

To produce openrsync the paper was very helpful, but also some reverse
engineering was required: ktrace, 'rsync -vvvv', and lots of tcpdump. In
hindsight a specification in the form of a IETF RFC would've been very
helpful :-)

Currently the closest thing to a specification about what happens on the
wire probably is the notes compiled during the openrsync development,
the notes can be found here (mandoc format): https://github.com/openbsd/src/blob/master/usr.bin/rsync/rsync.5

For your convenience i put a HTML version up here: http://sobornost.net/~job/rsync.5.html

Kind regards,

Job

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Nov 2020 12:25:48 -0500

Roman, Keith,

Two small additions to parts of your recent comments...

--On Thursday, November 26, 2020 14:39 +0000 Roman Danyliw
<[email-masked]> wrote:
>...
> Unlike John, we had discussed a broad array of content, so I
> also want to point out that the ftp.rfc-editor.org site has
> different content that ftp.ietf.org -- RFCs and I-Ds are
> there.  However, for example, the mailing list archives and
> charters are NOT on the rfc-editor site.

I am personally not very concerned about FTP access to mailing
list archives (which are becoming less useful as some efforts
are shifted to github and similar "living" repositories) or
charters.  Personally, I'd be happier if we took "expired"
seriously such that, after some reasonable grace period, such
I-Ds were part of a separate repository and required
significantly more effort to get to than clicking on a link in
the datatracker that says "A copy of the expired Internet-Draft
can be found at...".   However, I'm quite sure after other
rounds of discussions that I'm in the rough about that.  I would
also be happier if there were a good mechanism (whether via
HTTP, rsync, or FTP that make it convenient to retrieve current
versions only (neither FTP --including LIST or NLST and local
grep-- nor rsync are good at that).  Consequently, if an FTP
repository consisted only of I-Ds that were current or that had
expired, without being replaced, more than, say, six months
previously, I'd be quite happy.  But I'm probably in the rough
about that too.

Keith's rather nice list of functionality available via FTP that
is not readily available through HTTP and the datatracker in
https://mailarchive.ietf.org/arch/msg/ietf/ukTEvRqPF-91riLLjyJEy20T2Q8
is relevant here.  But there is another tradeoff in that
situation.  For me (although I'm guessing not for Keith, Lyndon,
and some others), my need for FTP access to I-Ds would go down
another notch if, when I did a search in the datatracker and got
back a multiple-document list, it gave me checkboxes for format
preferences and to identify the documents I wanted and then
allowed me to click something to automagically retrieve/download
that collection of documents.  Today, I have to either pull up
the page for one document at a time, download or view it, and
then go back to the listing for the next document while, with
FTP, if some conditions are met, I can pull the lot in one
operation.  [1]

And...

--On Thursday, November 26, 2020 09:45 -0500 Keith Moore
<[email-masked]> wrote:

>...
> More broadly, this conversation has convinced me that IETF
> cannot be relied on to provide the tools that IETF
> participants need, and in fact that IETF management has become
> hostile to participation in IETF by some kinds of people.  
> So if IETF is going to continue to be a viable consensus-based
> standards-making organization, either the management needs to
> do a significant about-face in its attitude toward
> participation, or IETF participants need to take on the burden
> of providing those tools.

The problem with the latter is that, unless the participant is
part of an organization that is willing and able to provide a
long-term commitment, you end up with your archive, I end up
with my archive, Lyndon ends up with his, etc.  We might be
willing to share our archives with a few hundred of our closest
friends, but the arrangements are inherently unstable _and_
cause us, each the extra work and operational complexity "the
IETF" wants to shed.  That is why I asked the question long ago
about what the actual operational costs of the IETF running/
retaining the FTP service are (don't think I ever got an answer
that I could use):  If we had a number and it realistically
represented marginal costs, at least a few of us much be willing
to buy into it on a fee for service subscription basis,
considering such a service with long-term guarantees and an SLA
to be much cheaper (because of economies of scale if nothing
else) and more convenient than doing it ourselves.

More generally, people who believe the IETF has become hostile
to their participation have an option that you didn't mention.
That is to take the work they are interested in to other
organizations and standards bodies.  If those bodies consider it
in their interest (even if just to make their standards
relatively more credible), such people might help them identify
the perceived hostility of the IETF to other points of view or
ways of working and the consequent lack of broad perspective in
its standards-making activities.   I hope that, before getting
to point of taking such actions, the people who feel that way
have explained the problem to the Nomocm and that, if they did
not do so before the nominal cutoff at the end of the IETF week,
the Nomcom can be persuaded to listen.

best,
    john

[1]  It may or may not be relevant that ISO's TC eCommittee
mechanisms and the "livelink" mechanisms used by several
National Member Bodies as part of their secretariat functions
for TCs -- many of which would probably give a very blank look
it someone said "FTP" to them00 provide exactly that type of
functionality.  Speaking for myself only, I find it a tad
depressing that they are ahead of us.

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Nov 2020 13:24:02 -0500

Job,

Thanks for posting this -- I've often asked the same question
Dave did.  He asked for publication as an open standard; despite
the documentation you cited, there isn't one of those.  If there
is significant demand, I can't imagine it would be a huge amount
of trouble to take those two specs, convert them to the form of
a standard, and have the IETF process it.  It isn't as if there
are not widely deployed interoperable implementations.  The only
risk I'd see would be if the IETF could not resist "improving
things" by adding new features or otherwise.

FWIW, some years ago I discovered a native rsync client for
Windows, Mac, and iOS whose author claims the core of the client
was written from scratch.  I haven't stressed it very much (and
have not tried the non-Windows versions) but, AFAICT, it
interoperates cleanly with the GPL and OpenBSD versions you
list.  The implementation is, however, client-only and is a
commercial (licensed for money after a free trial) product.
https://acrosync.com/ if you are curious.

    john

--On Thursday, November 26, 2020 17:20 +0000 Job Snijders
<[email-masked]> wrote:

>> I appreciate this is slightly off-topic, but given the
>> consistent suggestion that "rsync" should replace FTP,
>> perhaps this is the best place to ask - I can find an
>> implementation of rsync easily enough, but I can't find the
>> specification published as an open standard anywhere - do you
>> know where I can find it?
> 
> Fun question!
> 
> The RSYNC algorithm is outlined in these (excellent) papers by
> Andrew Tridgell:
> https://rsync.samba.org/~tridge/phd_thesis.pdf and
> To produce openrsync the paper was very helpful, but also some
> reverse engineering was required: ktrace, 'rsync -vvvv', and
> lots of tcpdump. In hindsight a specification in the form of a
> IETF RFC would've been very helpful :-)
> 
> Currently the closest thing to a specification about what
> happens on the wire probably is the notes compiled during the
> openrsync development, the notes can be found here (mandoc
> format):
> https://github.com/openbsd/src/blob/master/usr.bin/rsync/rsync
> .5
>

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 26 Nov 2020 19:01:57 +0000

I kinda agree with this.

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 14:11:19 -0500

On 11/26/20 12:08 PM, John Levine wrote:

> the rsync: URI but refers back tohttp://rsync.samba.org/  for details
> on rsync.
>

I remember when IETF cared about making standards and promoting broad 
interoperability.

Keith

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 26 Nov 2020 14:59:10 -0500

> and then stuff does hit the fan.

I agree with you that this sort of cascade-of-failures event seems unlikely.

But I've learned a few hard lessons in the last couple of decades and one
of those is that removing our tried-and-true old bridges because new ones
make them putatively obsolete sometimes comes back to haunt us.

	[ A timely example has presented itself under unfortunate
	circumstances.	At this moment, the Baltimore County (Maryland)
	public school system's entire IT infrastructure is down due to an
	apparent ransomware attack.  They can't email students/parents
	about it...because they shut down the old system.  They can't
	call students/parents...because they shut down the old system.
	They can't text students/parents...because they shut down the
	old system.  They've had to resort to social networks (urgh)
	and local news media to get the word out.

	It would have cost them almost nothing to keep an old, separate
	email server with sufficient capacity and an appropriate contact
	database running on a disused desktop PC that was physically
	disconnected until needed.  But they didn't. ]

So if the incremental cost of keeping an old bridge around, even if it's
very lightly used, is small enough...then I think it's a good idea to
keep it.  Consider it an insurance policy against a low-probability event.

---rsk

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 26 Nov 2020 15:19:24 -0500

Hi,

I work for the Internet Society, but this is my personal opinion.

>	disconnected until needed.  But they didn't. ]

I think this FTP discussion and the above share something, which is a presumption that there are things that are just sitting around and that don't require any attention.  I think this is false, and I would like to suggest that just about everyone in this discussion knows that to be the case, but is forgetting it because the costs are externalized.  This isn't meant to be a criticism, but just to draw to attention an important consideration about who decides.

At a high level of abstraction, of course, FTP and the "old disused desktop PC that is physically disconnected until needed" have this quality: you just put them in place and ignore them until you need them.

Yet we all know, I hope, that services that are actually connected (as would be the case for the FTP service) do in fact consume resources.  If you're the sysadmin, every service you're operating is another threat vector and also another thing you have to test whenever upgrades are needed or migration has to happen.  It's another monitor you need to maintain, and another opportunity for the pager to go off.  Someone has to pay those costs, and the discussion about FTP ought to reflect, in fact, that there's a trade-off between those who want the service and those who are going to have to pay the maintenance costs.  (Emphasising that I'm speaking for myself, I'd find the case for maintaining FTP to be a lot more compelling if the FTP preservationists were offering to run it.)

Similarly, even though it is the inverse case, the system that has been unplugged long enough is rapidly shown to be worthless when it is put into use, unless it has been maintained, updates applied, and configuration changes related to the changing environment were kept up to date.  Heaven help the shcool administrator who decided on a best-effort "keep the old PC around" strategy that had old contact data, contact data for former students, and that was missing new students' contact data.  This is the reality behind the evergreen refrain, "Test your backups."

As a practical matter, if you don't treat things you plan to rely on in emergencies as a first-class part of your production environment, they're much worse than nothing. They're false promises. So the real question has to be whether the additional protection is worth the cost of providing it. Too often, the way that calculation is made is, "Is it worth it _to me_ to keep this thing given that the cost is borne by someone else?" Some of those costs are in dollars only in a notional sense, and we don't need to care about the costs because it's someone else's problem. But I don't think that is a reasonble way to evaluate the trade-off. We should evaluate the trade as though we ourselves are paying the cost and receiving the benefit, and we should be scrupulously honest with ourselves about what those costs are likely to be in practice.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 16:06:44 -0500

On 11/26/20 3:19 PM, Andrew Sullivan wrote:

>
> I think this FTP discussion and the above share something, which is a 
> presumption that there are things that are just sitting around and 
> that don't require any attention.  I think this is false, and I would 
> like to suggest that just about everyone in this discussion knows that 
> to be the case, /but is forgetting it because the costs are 
> externalized/.

+1 to pretty much all of your message except for this last bit (italics 
mine).    Just because we don't mention the costs doesn't mean we're not 
aware of them.   (Though so far there's /still/ been no estimate of 
those costs, so basically we're left to guess.   And that might be part 
of why we don't say much about them.)

I do question the model that says that the security risks are related to 
the number of TCP ports used rather than, say, the number of lines of 
code that are exposed to externally originated traffic.   Of course 
that's not a great model either because the quality of the code matters, 
the implementation language matters, the protocol design matters, etc.

Keith

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Nov 2020 17:00:24 -0500

--On Thursday, November 26, 2020 14:11 -0500 Keith Moore
<[email-masked]> wrote:

>> If you're wondering if there's an RFC for it, nope. RFC 5781
>> specifies the rsync: URI but refers back
>> tohttp://rsync.samba.org/  for details on rsync.
>> 
>> I don't suppose anyone would object if we made an RFC out of
>> the spec but given that the tech report, thesis, and multiple
>> implementations are widely available without restriction
>> (other than GPL on some of the code) it doesn't seem worth a
>> lot of effort.
> 
> I remember when IETF cared about making standards and
> promoting broad interoperability.

Of course, there would be another way to do this, one that would
create an RFC but not a standard.  One could cobble a document
together that described what rsync was all about in the abstract
and introduction, include the references that have popped up in
this thread (and probably including a lot of text by reference),
and then hand it over to the ISE for publication of a
description of a protocol that is widely used in the community
for the information of the community.  If the IESG then wanted
to take such a document over and classify it as standards track,
I presume no one would object as long as they did not create a
working group that had the goal of either hanging bags on the
side of the thing or improving it enough that it was not
interoperable with deployed implementations. 

But there is not much evidence, in this case, that anyone cares
even enough to do that... and, if no one wants to invest even
that level of effort, then I think we need to agree with John
Levine's conclusion.

    john

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Nov 2020 20:43:53 -0500

On 11/26/20 5:00 PM, John C Klensin wrote:

> create an RFC but not a standard.

Yes, it seems like an Informational document, but one that actually 
describes the rsync protocol in sufficient detail to implement it in a 
way that interoperates, would be a useful service to the Internet 
community.   And it should be Informational rather than standards-track.

(Note: It's very important that such a document be accurate.  I remember 
how much breakage RFC 1179 caused, and I suspect that it still continues 
to cause some.)

What I have a hard time with, given that we're supposedly a 
standards-making organization that's trying to promote interoperability, 
is that people who seem to be somewhat prominent in this organization 
are now arguing that we should replace a proven standard protocol that 
has probably hundreds of implementations, with a protocol that may be 
less functional, is not a standard, doesn't have a published spec, and 
only seems to have a small number of implementations.

(If someone isn't into promoting standardization and interoperability, 
why are they participating in IETF?)

Of course, publishing an accurate description of the rsync protocol 
would address at least part of that problem.

There's also the related argument that some people keep making that 
rsync is sufficient not because it's a widely or easily implemented 
protocol, but rather, because there's a program that implements rsync 
that people can run from scripts.   Apparently bits on the wire as a 
basis for interoperability isn't important any more and we can just go 
back to standardizing APIs?

And whatever happened to the notion that we should eat our own dog 
food?   (And if the dog food doesn't taste good, why don't we fix it?)

Keith

---

From: Dave Cridland <[email-masked]>
Date: Fri, 27 Nov 2020 10:49:05 +0000

On Fri, 27 Nov 2020 at 01:44, Keith Moore <[email-masked]>
wrote:

> describes the rsync protocol in sufficient detail to implement it in a way
> that interoperates, would be a useful service to the Internet community.
> And it should be Informational rather than standards-track.
>
> how much breakage RFC 1179 caused, and I suspect that it still continues to
> cause some.)
>
> standards-making organization that's trying to promote interoperability, is
> that people who seem to be somewhat prominent in this organization are now
> arguing that we should replace a proven standard protocol that has probably
> hundreds of implementations, with a protocol that may be less functional,
> is not a standard, doesn't have a published spec, and only seems to have a
> small number of implementations.
>
> (If someone isn't into promoting standardization and interoperability, why
> are they participating in IETF?)
>
> Of course, publishing an accurate description of the rsync protocol would
> address at least part of that problem.
>
> There's also the related argument that some people keep making that rsync
> is sufficient not because it's a widely or easily implemented protocol, but
> rather, because there's a program that implements rsync that people can run
> from scripts.   Apparently bits on the wire as a basis for interoperability
> isn't important any more and we can just go back to standardizing APIs?
>
> And whatever happened to the notion that we should eat our own dog food?
> (And if the dog food doesn't taste good, why don't we fix it?)
>
 This was, of course, where I was going with my rhetorical question.

More broadly, we seem to have - as a community, I mean - decided not to
bother with application standardization anymore.

I see this most painfully with XMPP, of course, but I also see no interest
from the vast majority of the community in standards around email, for
example, except where it matters to connect Google to Facebook.

I came to the IETF originally drawn by the work going on in FTPEXT, around
20 years ago. "Retiring" FTP in favour of a newer protocol would be
bittersweet, of course, but retiring it in favour of a broadly proprietary
(albeit open source) file transfer/sync protocol simply doesn't sit right.

Overall, the IETF seems to be sending a clear signal that application layer
interoperability is unimportant, and existing protocols should be retired,
replaced by open source projects or whatever proprietary solution seems
convenient and commonplace.

I understand the convenience, and I understand the need for a commonplace
solution.

But aren't we the very people who make the complicated and bespoke into the
convenient and the commonplace?

Dave.

---

From: tom petch <[email-masked]>
Date: Fri, 27 Nov 2020 11:02:47 +0000

> that level of effort, then I think we need to agree with John

Back in 2008, sidr chose rsync as the foundation of its protocol.  It 
referenced rsync.samba.org.  See, for example, RFC6480.

Tom Petch

> .

---

From: "John Levine" <[email-masked]>
Date: 27 Nov 2020 13:45:49 -0500

>> small number of implementations.

I think we have it backwards. On today's Internet, rsync is clearly a
lot more useful than FTP. It doesn't have FTP's quaint warts (telnet
compatibility and in-line port numbers) and is better suited for large
disks and fast networks. The relative amount of traffic we see
confirms this.

This tells me not that we should stick our heads in the sand and
pretend that if FTP was state of the art in 1985 it must still be now,
but rather that since we all use rsync, it's worth at least
documenting and if possible standardizing.

I said it's not urgent because there is a lot of public documentation
and open source code that is not going away, so there is no chance
that it will become impossible or even inconvenient to use rsync, but
that also means we have a good place to start if we want to publish a
spec.

R's,
John

---

From: Joseph Touch <[email-masked]>
Date: Fri, 27 Nov 2020 11:36:50 -0800

> On Nov 27, 2020, at 10:45 AM, John Levine <[email-masked]> wrote:
> 

FWIW, although that may be true for this use case, it’s useful to keep in mind that FTP encodes a large body of experience that, in some ways, HTTP continues to try to evolve to replicate (like letting IP be the multiplexing unit and automatically avoiding HOL blocking).

FTP isn’t quaint in that regard. It remains the gold standard in many ways and has many lessons still to offer.

(Lessons; not necessarily needed operationally)

Joe

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 27 Nov 2020 19:56:21 +0000

Note that I am *not* commenting on the proposal to remove FTP.  Joe’s note just brought up some questions.

  *   FWIW, although that may be true for this use case, it’s useful to keep in mind that FTP encodes a large body of experience

I thought FTP stopped evolving in like 2005 with RFC 4217, about how to use TLS.  Am I missing something?

  *   that, in some ways, HTTP continues to try to evolve to replicate (like letting IP be the multiplexing unit and automatically avoiding HOL blocking).

Does FTP do multiplexing?  How?  Which RFC?

  *   FTP isn’t quaint in that regard. It remains the gold standard in many ways and has many lessons still to offer.

Such as?

---

From: Joseph Touch <[email-masked]>
Date: Fri, 27 Nov 2020 12:29:14 -0800

> On Nov 27, 2020, at 11:56 AM, Salz, Rich <[email-masked]> wrote:
> 
> FWIW, although that may be true for this use case, it’s useful to keep in mind that FTP encodes a large body of experience
>  

Everything that HTTP continues to reinvent that was in FTP before there was an HTTP.

> that, in some ways, HTTP continues to try to evolve to replicate (like letting IP be the multiplexing unit and automatically avoiding HOL blocking).
>  

Yes; it opens a connection per exchange and let’s IP multiplexing do its job. As far back as the mid-90s I noted that HTTP was headed towards reinventing that muxing at the app layer and it now does. That creates a problem of layered multiplexing and how the two interact, esp. for DSCP tagging, among other things.

> FTP isn’t quaint in that regard. It remains the gold standard in many ways and has many lessons still to offer.

Don’t reinvent multiplexing, for one. Have a common data encoding or support translation. Separate the control channel from the data channel. Support transfer restart. Support parallel transfers without HOL blocking. Support transfers with record boundaries.

Some people, IMO, incorrectly claim that HTTP fixed the bugs in FTP; AFAICT, it’s the been converse.

Joe

---

From: Keith Moore <[email-masked]>
Date: Fri, 27 Nov 2020 15:52:55 -0500

On 11/27/20 1:45 PM, John Levine wrote:

> lot more useful than FTP.

You're comparing apples to hammers.   And frankly, it's insulting.

Keith

---

From: "John Levine" <[email-masked]>
Date: 27 Nov 2020 20:54:11 -0000

In article <[email-masked]>,
Joseph Touch  <[email-masked]> wrote:
>Yes; it opens a connection per exchange and let’s IP multiplexing do its job.

Web browsers were doing multiple http requests in parallel in the
1990s. I said something about it in an early edition of Internet for
Dummies.

This doesn't strike me as a particularly interesting aspect of FTP.

---

From: Joseph Touch <[email-masked]>
Date: Fri, 27 Nov 2020 13:04:14 -0800

> On Nov 27, 2020, at 12:54 PM, John Levine <[email-masked]> wrote:
> 
> 1990s.

Largely to get around TCP slow start as a hack at the time - not to rely on IP multiplexing.

I.e., HTTP  still had HOL blocking because it tried to reuse a connection for multiple xfers. Not the same thing as exclusively relying on IP multiplexing. 

We discussed this in several webmux meetings in the IETF back then.

Joe

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 27 Nov 2020 21:53:20 +0000

*   Everything that HTTP continues to reinvent that was in FTP before there was an HTTP.
That’s not an answer, of course, but okay.  I guess you meant the list below.

  *   Yes; it opens a connection per exchange and let’s IP multiplexing do its job. As far back as the mid-90s I noted that HTTP was headed towards reinventing that muxing at the app layer and it now does. That creates a problem of layered multiplexing and how the two interact, esp. for DSCP tagging, among other things.
The TCP and TLS handshake costs are seen as prohibitive.  I guess not for FTP which isn’t used in e-commerce much.  Might be interesting to code up an FTP script that fetched Amazon’s homepage via TLS. But hey, let’s throw out TCP fastopen and TLS earlydata, too.

  *   Don’t reinvent multiplexing, for one. Have a common data encoding or support translation. Separate the control channel from the data channel. Support transfer restart. Support parallel transfers without HOL blocking. Support transfers with record boundaries.
Multiplexing, see above. Common data encoding seems to be mime-types and Accept headers. Restart and record boundaries seem to be covered by byte ranges.  Control channel is a neat idea, I guess that’s QUIC’s stream zero. Or BEEP’s channel zero.  I liked BEEP, too bad the Internet didn’t.

---

From: Joseph Touch <[email-masked]>
Date: Fri, 27 Nov 2020 14:05:45 -0800

> On Nov 27, 2020, at 1:53 PM, Salz, Rich <[email-masked]> wrote:
> 
>  
> The TCP and TLS handshake costs are seen as prohibitive.

They could easily have been handled - e.g., by “priming” additional connections in parallel. I grant that FTP has not evolved in a while and has issues - which could be fixed - but aren’t.

Again, I’m not suggesting using FTP, but it’s inaccurate to claim that FTP is some sort of outdated precursor to HTTP in many ways.

>   I guess not for FTP which isn’t used in e-commerce much.  Might be interesting to code up an FTP script that fetched Amazon’s homepage via TLS. But hey, let’s throw out TCP fastopen and TLS earlydata, too.

As I said, HTTP keeps trying to “keep up” in many ways (not all, but many) with what FTP had decades ago.

There’s still lessons to be learned there. Those who don’t study history are doomed to repeat it, I suppose.

Joe

---

From: tom petch <[email-masked]>
Date: Sat, 28 Nov 2020 12:49:44 +0000

Retiring the FTP service will impair my ability to contribute to the IETF.

For a long time, I struggled to download copies of IETF documents in the 
form that the author intended.  The topic arose on this list and I was 
pointed at the FTP service and this solved my problems; the amount of 
work I could do in the IETF increased.

FTP is so simple to use, a list of what I need to know with no 
extraneous information or fancy formats from which I can drag and drop 
or open and save; and which AFAICT accurately reproduces what is there 
on the FTP site; and I can do it all with a mouse, no keying needed, 
always a gain. Drag and drop preserves the original date which is 
valuable as I can use that to order work.

The availability is good; I cannot recall when last I could not access 
it.  With HTTP/HTML, there are days every week when my access is 
impaired and I have to try again later.  Again, this has been discussed 
on this list and the suggestion was that middleware, such as Cloudflare, 
was to blame.  Keep trying and it works but when I am reviewing an I-D 
with 47 somewhat elderly references in it, then the hassle becomes too much.

I have tried the URL given elsewhere in this thread.  I know that with 
.pdf, on (almost) every website, if I open and save, then Acrobat Reader 
will tell me that the file is damaged and cannot be repaired.  Rather, I 
must Right Mouse Click, 'save target as' and will get a message that the 
save has failed except that the file is now on my PC and Acrobat Reader 
is happy with it.

Using this technique and the given URL, each line has been slit in two, 
neatly indented, as if the server has decided what size window I will 
use in future and has reformatted the document accordingly.

Trying a separate technique of open and save, then all ASCII control 
characters have been removed to leave a Unix style stream of 86kbyte, 
equally useless.

Since HTTP/HTML is used by almost everyone to access almost everything, 
I imagine that there is an army of workers constantly seeking to 
'improve' the user experience.  Thus with another, non-IETF service, I 
have to logon but when I came to logoff, the other day, nothing 
happened.  I dislike leaving secure connections dangling so spent what 
became several hours trying to close the connection and found that when 
I clicked on signout, I could Right Mouse Click, get a drop down menu in 
which the 'Open' option closed the connection. Doubtless some web 
designer is feeling satisfied at having 'improved' the user experience 
but, for me, it took up the time I as going to spend on the QUIC I-D, 
then in Last Call.

Or again, I resized the window I use and when next I restarted the PC I 
was given the shrunken window with a message that (essential to me) 
fields were supressed because they would not fit.  I resized the window, 
carefully shut down, rebooted and got the shrunken window.  This 
persisted for days.  HTTP/HTML seems to have decided that because I had 
changed the size once, then anything else must be a mistake.  Another 
triumph for a web designer improving the user experience.

All this experience of HTTP/HTML is on shrink-wrapped Windows, 
factory-installed on a HP PC, accessing the Internet via a ISP-mandated 
ADSL modem.

Public services on a local authority PC are no better. I wanted to 
review a backlog of e-mail that had built up and every time I started to 
scroll down the list, wanting then to work forwards in time, I was taken 
back to the top.  I worked out that this was caused by a new e-mail 
arriving, a common occurence on several IETF lists.  I raised an 
incident and was told that the system was working as designed; when a 
new e-mail arrives, you MUST stop what you are doing and look at it. 
Another pat on the back for a web designer, another barrier to work.

Just as a hammer knows that everything is a nail, so a HTTP/HTML service 
knows that everything is a web page and 'improves' it accordingly, to 
the detriment of getting work done.  I cannot recall when I have ever 
had a comparable problem with the IETF FTP service.

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Sat, 28 Nov 2020 17:47:18 +0100

> work I could do in the IETF increased.

Ok, I'll bite.

Are you saying that when you open

   https://www.rfc-editor.org/rfc/rfc7230.txt

in a browser and do a "save as...", you're getting a broken document?

Best regards, Julian

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 28 Nov 2020 18:00:52 +0000

> form that the author intended

Why?  I have to admit that this makes me giggle, like seeing the Original Words in gilt lettering.  But my question is serious.

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Nov 2020 13:48:58 -0500

--On Saturday, November 28, 2020 18:00 +0000 "Salz, Rich"
<rsalz=[email-masked]> wrote:

>     > For a long time, I struggled to download copies of IETF
> documents in the     > form that the author intended
> 
> Why?  I have to admit that this makes me giggle, like seeing
> the Original Words in gilt lettering.  But my question is
> serious.

Tom,

While I am concerned about the rest of your comments, especially
if the txt and PDF copies on the IETF site are not working for
you [1], I have to share Rich's concern about "form the author
intended".  If you are having that problem with RFCs and go far
enough back, the only way to get the original (and presumably
forms involve getting access to archives of paper copies that, I
assume, are still at ISI, and getting photocopies.  Even for
RFCs that were available in machine-readable form a few years
later, you would have to retrieve Postscript files and render
them.  I don't have any idea how stable Postscript rendering
mechanisms (and their supposed clones) were over time and across
devices or whether they were "improved" enough to make assuming
they represented what the author intended questionable. [2]

For I-Ds, my memory may be incorrect but I believe that, for
years, the only forms that could be submitted were text (not
even nroff - one needed to do the formatting and submit the
text).    The rules about the text files were, again IIR, rather
relaxed.  There were likely limitations about line widths but,
e.g., pagination was certainly not required.  At least in my
case, those text-file submissions (whether prepared in a text
editor like emacs, in some runoff descendant, or in WordPerfect
or WordStar), were often compromises between the form that I
intended, what was feasible, and how much time I was willing to
spend fussing.  To retrieve what I intended, you'd need to be
able to retrieve hand-written notes (if they existed) or read my
mind back several decades.  So, while Rich giggles and I wonder
about media preservation, I'll assume that was not really what
you meant ... and am now wondering what you did mean.

    john

[1] I hope you have filed, or will immediately file, trouble
tickets on that with enough details that the issues can be
tracked down and repaired because, IMO at least, they represent
serious bugs somewhere.

[2] Perhaps Larry or someone else with long-term Adobe
experience could help with that if it were actually important or
people were interested.

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Nov 2020 13:54:29 -0500

--On Saturday, November 28, 2020 17:47 +0100 Julian Reschke
<[email-masked]> wrote:

>> documents in the form that the author intended.  The topic
>> arose on this list and I was pointed at the FTP service and
>> this solved my problems; the amount of work I could do in the
>> IETF increased.
>> ...
> 
> Ok, I'll bite.
> 

Julian,

In the hope of being clear about what we are talking about and
given Roman's repeated reassurances that none of the plan to
retire the IETF's FTP serices has anything to do with the RFC
Editor, would it be correct to say that you meant to ask about
the IETF's I-D collection, RFC mirror collection, and assorted
other documents available from the IETF to IETF tools sites?

I'd almost equally or more chagrined if the RFC Editor site was
producing broken documents, especially broken TXT documents (see
note sent a few minutes ago), but they are not the same question.

    john

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 28 Nov 2020 20:43:18 +0100

On 2020-11-28, at 13:49, tom petch <[email-masked]> wrote:
> FTP is so simple to use

Many of us have that relative that has a computer and has managed to get value out of it, discovering workflows on the way that actually work for them.  Say, to put a picture into an ms-word document, print it out and then scan it.  One might want to make a feeble attempt to suggest a more efficient workflow, but what they have come up with is now so ingrained that they will go back to the way of doing things they have discovered and practiced extensively.

Until the scanner breaks.

Grüße, Carsten

---

From: Keith Moore <[email-masked]>
Date: Sat, 28 Nov 2020 14:49:37 -0500

On 11/28/20 2:43 PM, Carsten Bormann wrote:

> One might want to make a feeble attempt to suggest a more efficient workflow,

Those who try actually using FTP, rather than merely disparaging others 
who do, might find that for some purposes FTP actually is more efficient.

Using the right tool for the particular job is generally a sign of 
wisdom and experience.

Keith

---

From: "John Levine" <[email-masked]>
Date: 28 Nov 2020 15:17:04 -0500

In article <9E191D6ECFE4BC432E78DEE7@PSB> you write:
>they represented what the author intended questionable.

Postscript has always been well defined, in the original Adobe
reference in 1985 and then the much expanded one in 1990.  Some
of the PS RFCs may have depended on non-standard fonts but that all
seems to be sorted out in the PDFs created from them.

I looked at a few of the first PS RFCs from 1989 in the viewer on my
Mac and they all look fine.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Nov 2020 16:31:03 -0500

--On Saturday, November 28, 2020 15:17 -0500 John Levine
<[email-masked]> wrote:

>> mechanisms (and their supposed clones) were over time and
>> across devices or whether they were "improved" enough to make
>> assuming they represented what the author intended
>> questionable.
> 
> of the PS RFCs may have depended on non-standard fonts but
> that all seems to be sorted out in the PDFs created from them.
> 
> I looked at a few of the first PS RFCs from 1989 in the viewer
> on my Mac and they all look fine.

Good to know and not surprising... but "looks fine" is not
necessarily sufficient to satisfy Tim's criterion of "intent of
the author".  My personal guess is that particular criterion
--at least as I, and I think Rich, interpreted it is basically
impossible.  So the key question is whether Tim intended that in
a way that is plausible and can be better defined or whether we
can (and should) just move on.

   john

---

From: Julian Reschke <[email-masked]>
Date: Sat, 28 Nov 2020 23:10:28 +0100

> other documents available from the IETF to IETF tools sites?

Actually, I asked about RFCs on rfc-editor.org - which is clearly the
point to go for authoratative versions of RFCs.

I'm just trying to find out whether something is broken with respect to
http(s) access as opposed to ftp service - be it on ietf.org or
rfc-editor.org. If there is something indeed broken, we should fix it,
right?

Best regards, Julian

---

From: "John Levine" <[email-masked]>
Date: 28 Nov 2020 17:17:35 -0500

In article <541CBB9D85D6EEF8F3D05D51@PSB> you write:
>impossible. 

Well, sure, without a time machine there's no way to tell what someone
saw on his screen or page 30 years ago, but I would be pretty amazed
if what people saw now was any different from what they saw then,
other than the font substitutions I mentioned. Adobe is still around,
they made the most widely used Postscript rendering software then and
still do now, and they take stability of the spec very seriously.

Even ASCII has its bitrot aspects. As I'm sure you recall, the
character ^ that now looks like a circumflex sometimes used to be an
uparrow, and _ which now looks like an underline was often a left
arrow.  But we seem to be OK with old ASCII documents anyway.

R's,
John

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 28 Nov 2020 23:51:59 +0100

On 2020-11-28, at 23:17, John Levine <[email-masked]> wrote:
> 
> Even ASCII has its bitrot aspects. 

As does the way we interpret certain characters from ASCII that haven’t changed in the mid-1960s...

https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html

Grüße, Carsten

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Nov 2020 18:03:46 -0500

--On Saturday, November 28, 2020 17:17 -0500 John Levine
<[email-masked]> wrote:

> character ^ that now looks like a circumflex sometimes used to
> be an uparrow, and _ which now looks like an underline was
> often a left arrow.  But we seem to be OK with old ASCII
> documents anyway.

IIR, not quiet, but I don't have time to dig out my copies of
the first and second versions of ASA/.../ANSI X3.4 and we are
already more than far enough off-topic.  

Where I'm confident we can agree is that it is wiser to rely on
a stable page image format (like PDF and ideally with no
external references) than on a print specification format (like
Postscript) to be sure that what we are seeing today and what
the author saw are the same.  I think we also agree that,
between Postscript and PDF, the differences are likely to be
small (you point out the possible font issues with the former
and the possibility of external references as well as font
issues are why the effort that produced the PDF/A specifications
came into being.  I presume we would also both agree that either
one is more likely to reproduce what the author saw than almost
any sort of textual markup language in the absence of external
specifications or assumptions or of a simple stream of ASCII
characters (given that ASCII, like Unicode, associated abstract
characters and not particular glyphs with code points/values).

The question I've been trying to get at, and won't have an
answer to until we hear from Tim, is whether what he meant by
what "the author intended" is actually what you are describing
or whether he was really talking about intent.  I think what you
are talking about would be closer to "what the author saw when
the document was first made available".  Even then there is a
sometimes-significant difference between some original RFCs and
the results of the conversion project to get RFCs into
machine-readable format.  For some of the RFCs covered by that
project, that project lost assorted appendices and illustrations
that the authors certainly intended to be present.

So, again, I'm not sure exactly what we are talking about -- and
even, given Julian's question, whether we are talking about RFCs
in the RFC Editor repository, anything else the RFC Editor may
be mirroring, or I-Ds, RFCs, and other materials in IETF
repositorites.

    john

---

From: Larry Masinter <[email-masked]>
Date: Sat, 28 Nov 2020 15:11:16 -0800

I don't know about the old PDFs of RFCs scanned from paper, but at least
the ones being generated now are not just "PDF" but "PDF/A" (at least
that's what RFC 7995 called for).

*PDF/A* is an ISO
<https://en.wikipedia.org/wiki/International_Organization_for_Standardization>-standardized
version of the Portable Document Format
<https://en.wikipedia.org/wiki/Portable_Document_Format> (PDF) specialized
for use in the archiving <https://en.wikipedia.org/wiki/Archive> and
long-term preservation <https://en.wikipedia.org/wiki/Digital_preservation>
 of electronic documents <https://en.wikipedia.org/wiki/Electronic_document>.
PDF/A differs from PDF by prohibiting features unsuitable for long-term
archiving, such as font <https://en.wikipedia.org/wiki/Font> linking (as
opposed to font embedding <https://en.wikipedia.org/wiki/Font_embedding>)
and encryption.[1]
<https://en.wikipedia.org/wiki/PDF/A#cite_note-nutshell-2-1-1> The ISO
requirements for PDF/A file viewers
<https://en.wikipedia.org/wiki/File_viewer> include color management
<https://en.wikipedia.org/wiki/Color_management> guidelines, support for
embedded fonts, and a user interface for reading embedded annotations
<https://en.wikipedia.org/wiki/Annotation>.

Of course, you can declare anything to be there for "the long term" and
lose because of disuse, but the widespread adoption of PDF/A as a long term
archival format increases the likelihood of interpretability.

On the other hand, there is no SVG/A (the analog of PDF/A but for SVG. This
means that the likelihood that, say, 50 years from now, you won't be able
to find a compatible renderer for IETF based RFCs in HTML+SVG is (IMO)
greater.

I'm a fan of preserving software through emulation (engaged in preserving
some old software from the 80s) but I think to be careful it might be
important to archive the Tools implementations of various versions of the
RFC XML formatted. Also when regenerating the text, PDF, HTML derived
forms, being careful to archive the previous renditions (as a safety
measure).
-- 
https://LarryMasinter.net  https://interlisp.org

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 28 Nov 2020 20:58:29 -0800

> of why we don't say much about them.)

Let's also remember that costs need not always be measured in dollars.

Given that this whole adventure started with the tools team asking
Roman/the IESG to run a community consensus process, my operating model is
that there is a morale cost to having the team (volunteers and staff alike)
put the work in to move the FTP service over to the new generation of
infrastructure and continue running it when "nobody uses it".  I'm happy to
be proven wrong by better data, but would prefer to not be treating these
individuals as mindless cogs in a machine.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 28 Nov 2020 21:25:42 -0800

> part of those making the insults.

Agreed that the insults weaken the argument.

> which tools people should be using.

I continue to disagree with the phrasing here: the proposal at hand does
not "decide what tools people should be using".  It is an effort to gauge
consensus in the IETF community as to what services should be offered by
IETF infrastructure.  The existing infrastructure (not under discussion for
modification or removal) provides more than adequate facility for someone
else to provide their own mirroring and services for accessing the content,
so to say that the IESG is somehow going to decide that you shouldn't use
FTP at all does not follow.  If consensus is determined to be that the IETF
should stop offering FTP access, it means just that: *the IETF* is not
going to offer FTP service, but someone else is more than welcome to do so
if they find value in it.

> This is not their job,  not what 

Distraction from the main role, sure (but ietf@ is a recurring such
distraction).  Judging IETF consensus, though, is squarely in the job
description and as such seems to be something that we are chosen for.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 28 Nov 2020 21:45:06 -0800

> best assume that some will be.   

I think this sentiment bears reiterating, so +1.
Thank you for putting it to words.

> due to IESG refusal to enforce naming conventions:  When naming

Could you remind me of which deviations from
draft-(ietf|lastname)-wgname-slug are problematic for your workflow?  I
don't recall a particular IESG discussion on "let's not enforce naming
conventions" (and do recall some discussion about preserving some namespace
prefixes), but perhaps my memory needs a jolt to get in gear.

Thanks again,

Ben

---

From: Masataka Ohta <[email-masked]>
Date: Sun, 29 Nov 2020 16:32:24 +0900

Benjamin Kaduk wrote:

> Roman/the IESG to run a community consensus process,

The obvious solution to reduce our operational effort is to
deprecate the team.

> I'm happy to

Recognizing collective stupidity of a team is different from
insulting individuals consisting the team, though upper layer
people who improperly set up the team may be condemned.

						Masataka Ohta

---

From: "Salz, Rich" <[email-masked]>
Date: Sun, 29 Nov 2020 15:42:34 +0000

>    Given that this whole adventure started with the tools team asking
    Roman/the IESG to run a community consensus process

When I posted "I kinda agree with Keith"[1] I didn't know (or had forgotten) that this is how it started. I was wrong.

[1] https://mailarchive.ietf.org/arch/msg/ietf/PYgJqC2qmYcRswWiOmwyad9xheQ/

---

From: George Michaelson <[email-masked]>
Date: Mon, 30 Nov 2020 09:27:09 +1000

In hindsight, this has been a problematic decision. rsync is highly
exploitable for DOS and unexpected outcomes. Pragmatically it got us
somewhere. Its a bad fit for the CDN world we live in now.

https://www.ietf.org/proceedings/89/slides/slides-89-sidr-6.pdf

https://blog.apnic.net/2020/10/27/rpki-qa-the-trouble-with-rsync/

RPKI (sidrops) is trying to define a delta protocol up into more
normative state, and a deprecate-rsync draft is in hypothesis.

-G

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 12:30:53 +0000

Yes.

This is not the URL I used before, that came from a post from Roman and 
was for an older RFC.

Be clear, I want ASCII text, which I can view anywhere, with a wide 
range of tools, edit (well not often RFC) and will usually be what the 
author created.  So given the choices with 'save as' I will choose text 
and then the saved file starts with the complete CSS  (gee thanks web 
designer, how is it you always know better than me what it is that I 
want:-)  The other three options all give me different forms of the 
HTML.  So yet another way for HTTP/HTML to not give me what I want!

Tom Petch

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 12:37:51 +0000

> Why?  I have to admit that this makes me giggle, like seeing the Original Words in gilt lettering.  But my question is serious.

Rich

This is going back some years now and I do not recall in what way the 
documents were corrupt but it would likely be to do with the format, 
such as not respecting the ASCII control characters such as I got last 
week when an RFC was turned into a stream without formatting.  I said in 
my first post that the ASCII control characters has been elided and that 
is an overstatement - I have not looked at the hex to see if they are 
present, just noticed that it is as if they were not longer there, no 
new lines, no new pages, running heading and footing all forming a 
stream with the content. That was RFC282 (yes two hundred)

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 13:43:26 +0100

>> in a browser and do a "save as...", you're getting a broken document?

Well, in that case we should figure out what's going on. This is the
official plain text file published by the RFC Editor. If you can't
retrieve and save it without getting broken content, something is
seriously wrong with either your client or your network connection.

> This is not the URL I used before, that came from a post from Roman and

It would help if you provided a concrete example HTTP(S) URL for an RFC
(or an I-D) that fails for you (and optimally a description that the
failure exactly is).

> range of tools, edit (well not often RFC) and will usually be what the

Yes. That's the case with the text/plain resource mentioned above.

> designer, how is it you always know better than me what it is that I

That would be the case if you save an *HTML* resource, not a plain text
resource.

> HTML.  So yet another way for HTTP/HTML to not give me what I want!

What "other three options"?

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 12:50:12 +0000

John

Yes, my carefully crafted e-mail was not as precise as I thought.  I 
only got involved with the IETF in 1994 and the oldest document I have 
dates from 1997 so what I was referring to was the electronic, ASCII 
format and I was assuming that what made it onto the IETF site was what 
the author intended, perhaps after some iterations and assuming, as with 
any publisher, that the author accepts the house style of the publisher 
even if as a necessry evil:-)

Tom Petch

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 12:57:19 +0000

> Until the scanner breaks.

Yes, I know, my use of Windows likely puts me in a very small minority 
of those who contribute to IETF lists, while my preference for Lotus 
over Microsoft for anything office-related likely makes me a minority of 
one.  Keith's posts on this thread, like this note of yours, makes it 
clear how many different work flows we have.

But for me there is a difference between e.g. having Microsoft eliminate 
the MUA as they did 12 months ago and the IETF eliminating FTP.  The 
former was a shock, the latter seems unreasonable.

Tom Petch

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 30 Nov 2020 14:25:02 +0100

On 2020-11-30, at 13:37, tom petch <[email-masked]> wrote:
> 
> not respecting the ASCII control characters such as I got last week when an RFC was turned into a stream without formatting

This is getting a bit confused.

Recent RFCs (from RFC 8650 on) have not been ASCII, even in the plain-text version.
Even if the author’s text was ASCII-only Unicode, upon publishing a UTF-8 BOM is added.
(I think that is wrong, but that is a separate discussion, and I’m obviously in the rough here.)

Also, recent RFCs in plain-text version have not been paginated, which maybe what you call a “stream”.
(I also think that is suboptimal, but that is yet another discussion again.)

None if this is dependent on the mode of access.

Note that Web browsers might be adding BOMs or might be changing their behavior based on the presence of a BOM, which may be part of the unexplained effects you might be seeing.  Preferably, don’t use Web browsers or other sources of intelligent behavior.

Grüße, Carsten

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 09:19:11 -0500

> Note that Web browsers might be adding BOMs or might be changing their behavior based on the presence of a BOM, which may be part of the unexplained effects you might be seeing.  Preferably, don’t…

I learned a long time ago that if I want to print an (older) RFC or I-D 
from a web browser, the browser will delete the form feeds between pages 
even if it preserves the remainder of the text.  So if I want to print 
an RFC, I download the RFC in PDF format and print that.

If I just want to read the text, I might view the HTML.   But until 
recently the HTML versions have been so bad that I preferred to read the 
text versions.  The more recent versions of HTML from the RFC Editor are 
better.

If I want to edit an RFC I'll probably go looking for the xml 
versions.   If I actually want plain text I'll "download" (not view) the 
.txt version and edit it in emacs, which last I knew didn't get confused 
by BOMs and explicitly showed the characters in front of the file where 
they're easily deleted.  (But this might have changed, I haven't been 
bitten by BOMs in RFCs yet)

If I want a copy of the ASCII text of a single RFC I'll often right 
click and select download from a web browser.

If I want multiple RFCs of any type I'll fire up an FTP client.

But yeah, web browsers get in the way of a lot of reasonable things.  
IMO, if you view a file and then select "Save" it should save the 
original version of the file and not the HTML version that it converted 
the file into for convenience of displaying it.   But I gave up on 
browsers being reasonable a long time ago.

(Note that all the browsers I know/knew that still support(ed) ftp 
are/were agnostic about how the file was obtained - they would apply the 
same damage to a .txt file downloaded via ftp as to the same file 
downloaded via http.)

Keith

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 15:23:46 +0100

> original version of the file and not the HTML version that it converted

Over here, Firefox saves the originally received text file, so does Chrome.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 09:27:40 -0500

> Over here, Firefox saves the originally received text file, so does 

Perhaps that behavior has changed.   When you learn a way to work around 
damage, you tend to keep doing that even after the damage gets fixed.

---

From: [email-masked]
Date: Mon, 30 Nov 2020 06:36:38 -0800 (PST)

> damage, you tend to keep doing that even after the damage gets fixed.

True. but more to the point, are you willing to bet that this behavior is never
ever going to change in any future version?

If so, I have a couple of bridges you may be interested in purchasing at
Cyber Monday discount prices.

				Ned

---

From: Roman Danyliw <[email-masked]>
Date: Mon, 30 Nov 2020 14:55:49 +0000

Hi!

> Sent: Monday, November 30, 2020 9:37 AM
> To: Keith Moore <[email-masked]>
> Cc: [email-masked]
> > Perhaps that behavior has changed.   When you learn a way to work
> > around damage, you tend to keep doing that even after the damage gets
> fixed.
> 
> If so, I have a couple of bridges you may be interested in purchasing at Cyber
> Monday discount prices.

I'm having trouble understanding the failure mode.  Is the issue at hand that when one uses a browser to download say https://www.rfc-editor.org/rfc/rfc7230.txt, a text file isn't being returned?  Furthermore, is the concern that browser vendors have changed their behavior over time when saving .txt files?

Roman

---

From: [email-masked]
Date: Mon, 30 Nov 2020 07:18:27 -0800 (PST)

> I'm having trouble understanding the failure mode.  Is the issue at hand that
> when one uses a browser to download say
> https://www.rfc-editor.org/rfc/rfc7230.txt, a text file isn't being returned? 
> Furthermore, is the concern that browser vendors have changed their behavior
> over time when saving .txt files?

The failure mode is that browser behavior is in a constant state of flux. Some
time ago you got HTML-ized text, yesterday you got text without the form feeds,
today you get the text file verbatim. Who knows what you'll get tomorrow?

And as Keith points out, the problem is significantly exacerbated by
experienced people not engaging in problematic behavior without really thinking
about it, leading them to assert that there are no land mines simply because
the path they have learned to follow avoids them.

				Ned

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 16:50:51 +0100

> today you get the text file verbatim. Who knows what you'll get tomorrow?

Is it really in "constant flux"? Pointers appreciated.

Best regards, Julian

---

From: Roman Danyliw <[email-masked]>
Date: Mon, 30 Nov 2020 15:52:45 +0000

Hi Ned!

> Sent: Monday, November 30, 2020 10:18 AM
> > > True. but more to the point, are you willing to bet that this
> > > behavior is never ever going to change in any future version?
> > >
> > > If so, I have a couple of bridges you may be interested in
> > > purchasing at Cyber Monday discount prices.
> 
> > I'm having trouble understanding the failure mode.  Is the issue at
> > hand that when one uses a browser to download say
> > Furthermore, is the concern that browser vendors have changed their
> > behavior over time when saving .txt files?
> 
> time ago you got HTML-ized text, yesterday you got text without the form
> feeds, today you get the text file verbatim. Who knows what you'll get
> tomorrow?

I'm sorry, I still don't following.  Clicking on what link did you get HTML-ized text and then text another time?  If one goes to the datatracker (say https://datatracker.ietf.org/doc/rfc7230/) and one clicks "plain text", is one not getting plain text?  If one visits, https://www.rfc-editor.org/rfc/rfc7230.txt, is a TXT not returned?

If one is clicking on txt, html, pdf, htmlize, or with errata (all of the flavors from the datatracker) and not getting txt, html, pdf, html and html, respectively, this is a bug.  Let's troubleshoot it when it happens.

Regards,
Roman

> experienced people not engaging in problematic behavior without really
> thinking about it, leading them to assert that there are no land mines simply
> because the path they have learned to follow avoids them.
> 
> 				Ned

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 10:53:53 -0500

> ever going to change in any future version? 

Not me.  That's why I try to avoid using web browsers to download 
anything for which I need integrity.

(though I generally find that "Download" from web browsers works fine, 
but even that fails more often - sometimes without any error message - 
than downloads using file transfer apps)

Keith

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 30 Nov 2020 10:55:03 -0500

> somewhere. Its a bad fit for the CDN world we live in now.

arguably, CDN doesn't matter at all here, we are only shipping around some small
number of small files per endpoint. CDN usage is really a
micro-optimization which
may be nice to have but isn't super relevant to operating the system as a whole.

what matters is a transport where there are programming libraries (in the
language(s) you choose to use for your system/tools) that report sane errors
which can be handled programmatically.

Shelling out to run your transport is ... hard to
debug/think-about-programmatically/repair. :(

Yes, rsync did and does provide a running system today.
Yes, rsync is also painful in these systems because failures are 'impossible' to
recover from (except by off/on-again)

---

From: Larry Masinter <[email-masked]>
Date: Mon, 30 Nov 2020 07:56:34 -0800

MIME Sniffing Standard (whatwg.org) <https://mimesniff.spec.whatwg.org/>

On Mon, Nov 30, 2020 at 7:54 AM Keith Moore <[email-masked]>
wrote:

> Not me.  That's why I try to avoid using web browsers to download anything
> for which I need integrity.
>
> (though I generally find that "Download" from web browsers works fine, but
> even that fails more often - sometimes without any error message - than
> downloads using file transfer apps)
>
> Keith
>
>
>

---

From: [email-masked]
Date: Mon, 30 Nov 2020 07:58:01 -0800 (PST)

You just heard from two people on this list - Keith and Tom - that things have
changed over time. If you don't believe them, say so in so many words.

But let's assume for a moment that there haven't been any changes. Let me
rephase my question: Would you implement something that depened on browser
file save behavior remaining unchanged long term?

				Ned

---

From: Larry Masinter <[email-masked]>
Date: Mon, 30 Nov 2020 08:08:54 -0800

I provided a link to MIME Sniffing Standard (whatwg.org)
<https://mimesniff.spec.whatwg.org/>
but in particular, there is no reliable way to obtain the HTML + SVG
rendering using a browser and "Save As".
The WPACK "working group" is working on another way that "Save As" might
work (rather than multipart/related) if they are successful, and if the
browsers decide to adopt it.

---

From: Roman Danyliw <[email-masked]>
Date: Mon, 30 Nov 2020 16:11:55 +0000

Hi Larry!

From: Larry Masinter <[email-masked]>
Sent: Monday, November 30, 2020 11:09 AM
To: Roman Danyliw <[email-masked]>
Cc: Ned Freed <[email-masked]>; [email-masked]; Keith Moore <[email-masked]>; [email-masked]
Subject: Re: Call for Community Feedback: Retiring IETF FTP Service

I provided a link to MIME Sniffing Standard (whatwg.org)<https://mimesniff.spec.whatwg.org/>
but in particular, there is no reliable way to obtain the HTML + SVG rendering using a browser and "Save As".
The WPACK "working group" is working on another way that "Save As" might work (rather than multipart/related) if they are successful, and if the browsers decide to adopt it.

[Roman] Thanks for clarifying.  However, I don’t think that consistent HTML+SVG saving is the use case in question.  I believe it was that browsers were not consistently downloading text files.  As to the “save” vs. “save as”, I won’t presuppose the workflow of users.

Roman

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 30 Nov 2020 16:15:18 +0000

So you do it because something went wrong awhile ago, and downloading "source" avoided the problem and now it's ingrained into your workflow.

I'm not criticizing, after all our whole industry works that way. :)

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 11:21:42 -0500

On 11/30/20 10:52 AM, Roman Danyliw wrote:

> If one visits,https://www.rfc-editor.org/rfc/rfc7230.txt, is a TXT not returned?

How would you know?  If you visit a file from a browser, you only know 
what the browser shows you.

If for instance I click on that link and then hit ^U in the browser 
(which happens to be Brave), what I now see appears to be text with line 
numbers.   I can't tell if the formfeeds are there or not.

If I do the same in firefox, I get text without line numbers. The 
formfeeds actually appear to be there - they show up as squares with 
"000C" in them.

In chromium, I get the same behavior as Brave.

In the past, I've gotten other results, such as seeing HTML tags 
embedded in what is supposedly the "source" of the plain text file.

The general problem is that when you use a web browser to view something 
and then save or print it, the behavior is undefined. No standard says 
what should happen, and you don't know what you're going to get.

But a related problem is that people keep gratuitously changing things.  
So even if you find a workaround to some bit of damage, that workaround 
is not assured to work in the future.

To be clear, we're just talking about damage caused by web browsers 
here.   I doubt that anyone is clicking on a .txt and getting .html or 
.pdf from the web server.

But this illustrates why some of us prefer to avoid using web browsers 
for some things, and instead rely on tools that have well-defined behavior.

Also, I don't think the HTTP protocol corrupts files, though I have seen 
software that would silently ignore an incomplete HTTP file transfer.   
But if what you actually need to do is browse files to pick out the ones 
you want, vanilla HTTP doesn't provide what a tool needs to reliably do 
that.

Keith

---

From: Larry Masinter <[email-masked]>
Date: Mon, 30 Nov 2020 08:49:57 -0800

when I visit https://tools.ietf.org/id/draft-masinter-mime-web-info-00.txt
with three different browsers (Edge, Chrome, Firefox) only one (Firefox)
seems to have a menu that lets me save anything as a file: Firefox has a
"Save Page As" but I don't see a "Save". If I want to retrieve a page
without any helpful sniffing/transformation, I suppose I could use curl or
wget (curl vs Wget (haxx.se)
<https://daniel.haxx.se/docs/curl-vs-wget.html> (using
WSL on Windows? )
-- 
https://LarryMasinter.net  https://interlisp.org

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 17:01:58 +0000

> What "other three options"?

Julian

I keyed in the URL you gave above for RFC7230.  Using Internet Explorer 
(as installed by HP with Windows), 'Save as' gives me the four choices of

Webpage complete htm html
Web Archive single file mht
Webpage HTML only htm, html
Text File txt

from which I chose Text File.  Viewing the result with Wordpad gives me 
the CSS followed by the text.  The text is ok AFAICT but all that CSS is 
unwelcome and is what I referred to as broken.

File-Properties tells me that this is
Type HTML Document
which seems plausible with a
Created: 30/11/2020 (I am in a European Locale)
which is the same for all that I look at on the rfc-editor site via 
https: - accessing the IETF site File-Properties shows the date as
Not Available

I see no sign of Text/Plain (which, as you gather, is close to my heart:-)

Tom Petch

---

From: Larry Masinter <[email-masked]>
Date: Mon, 30 Nov 2020 09:02:03 -0800

> Is it really in "constant flux"? Pointers appreciated.
I sent one already:  MIME Sniffing Standard (whatwg.org)
<https://mimesniff.spec.whatwg.org/>

MIME SniffingLiving Standard — Last Updated 30 September 2020
-- 
https://LarryMasinter.net  https://interlisp.org

---

From: Roman Danyliw <[email-masked]>
Date: Mon, 30 Nov 2020 17:05:37 +0000

Hi Larry,

> From: ietf <[email-masked]> On Behalf Of Larry Masinter
> Sent: Monday, November 30, 2020 11:50 AM
> Cc: IETF <[email-masked]>
> when I visit https://tools.ietf.org/id/draft-masinter-mime-web-info-00.txt  with three different browsers
> (Edge, Chrome, Firefox) only one (Firefox) seems to have a menu that lets me save anything as a file: 
> Firefox has a "Save Page As" but I don't see a "Save". If I want to retrieve a page without any helpful 
> sniffing/transformation, I suppose I could use curl or wget (https://daniel.haxx.se/docs/curl-vs-wget.html
> (using WSL on Windows? )

Please see attached screen shot of Edge with "Save As" at the link noted above.

Regards,
Roman

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Mon, 30 Nov 2020 12:07:43 -0500

> changed over time. If you don't believe them, say so in so many words.

Changed once or twice over years or decades is not "constant flux".
Stepping back a bit, it seems to me that that one of the hidden
assumptions which has turned this into a long, drawn-out discussion,
is whether or not change should be tolerated, and over what time
scales.  It may very well be that for some people a change once a
decade is "contant flux".  In other cases, people may be willing to
accept some change every 2-4 years, so long as reasonable methods of
getting their work done are available, even if it does mean that they
have to adjust their workflow every few years.

Ultimately, if we have to support all workflows forever, then the job
of the people maintaining the tools and servies is going to either (a)
stagnate, by not adding new features, since that will increase their
maintenance load, or (b) grow without bound, as new features to ease
IETF participants' work are added, with no means of transitioning off
of older technologies.

I suspect the people who are so concerned about FTP overheads are
doing so for philosophical reasons, more than anything else.  For
while, when my preferred access method was over AFS, I was mirroring
FTP and I-D's to a local archive on an AFS cell at MIT.  It was *not*
a big deal, and if I needed to change the URL used to keep my local
mirror in sync (as I recall I needed to do once or twice over the 10
or so years), it really wasn't a big deal.  And disk space has gotten
cheaper over the years, so it *really* isn't that hard for people to
keep their own local mirrors if they really wanted to.

So I wonder if this whole, long, debate, is really more about people
who don't want to deal with any kind of change, because while *this*
change might be relatively easy to work around, the *next* one might
require a bit more work.  But the long-term question about how access
portals and other technologies should get retired is still going to
remain.

Perhaps if there was a deprecation window of, say, a year?  Maybe two
years?  This gives people *plenty* of time to investigate alternate
workflows and methods, and still allows for the secretariat and tool
teams to be able to continue to innovate without having to maintain
older mechanisms forever.

Cheers,

						- Ted

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:10:23 +0100

> changed over time. If you don't believe them, say so in so many words.

 From Tom I'm still waiting for a concrete reply. And what Keith says
hints at that this has not been working at one point in the past.

That does *not* mean it's in a constant flux".

> file save behavior remaining unchanged long term?

Honestly, right now I see no reason not to. In particular as there's
always curl as a fallback should there ever be a problem.

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 17:16:21 +0000

> (I also think that is suboptimal, but that is yet another discussion again.)

Carsten

Yes, we are on the same page here so I went back to RFC5347 which 
predates the recent changes and which has lost its layout.  This loss of 
layout used to be a common problem because of differing conventions over 
the use of ASCII CR-LF to separate lines, as opposed to just one of the 
ASCII control characters.  I do not have ready access to a hex editor 
and so cannot see if that is the case but it would not surprise me if 
the web browser, Internet Explorer, was doing something to the ASCII 
control characters; on the other hand, FTP is rich in options in this 
area, it being a significant issue at the time of publication of RFC959, 
so it just could be that FTP is getting it right whereas HTTP/HTML is 
not making any changes and renders me an unusable stream of bytes!

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:16:34 +0100

> rendering using a browser and "Save As".

Can you elaborate? As far as I understand, the HTML published by the RFC
Editor uses inlined SVG, so no packaging should be needed.

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 17:19:05 +0000

> damage, you tend to keep doing that even after the damage gets fixed.

Which is how I came to use FTP in the first place, it was the easiest 
way to get plain text ... well, as plain text!

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:25:53 +0100

> what the browser shows you.

Well, you could check with the web developer tools.

> numbers.   I can't tell if the formfeeds are there or not.

Firefox shows you the line feeds (as control char). So is your point
that there *can* be a user agent that behaves differently? How is that
materially different from different text editors doing things differently?

> what should happen, and you don't know what you're going to get.

<https://html.spec.whatwg.org/multipage/links.html#as-a-download> - but
I confess that I'm not going to read that algorithm :-)

I agree that Chrome does not *show* the FF character, but it *is*
present in the saved file.

> software that would silently ignore an incomplete HTTP file transfer.

Which *should* be a thing of the past with HTTP/2 because of better
message framing.

> you want, vanilla HTTP doesn't provide what a tool needs to reliably do

It does not?

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 17:27:15 +0000

> today you get the text file verbatim. Who knows what you'll get tomorrow?

All the CSS preceding the text; well, that is what I got this morning 
when prompted by Julian's post but tomorrow, well, tomorrow is another day!

As I just posted to Carsten, this was a major problem 35 years ago which 
is why protocols like FTP are rich in options to handle this.  I could 
be 180 degrees wrong that HTTP/HTML is making no changes and so giving 
just what is on the web site which is then no use to me and it is FTP 
that knows what I really want and is making corrections for me, such as 
inserting <CRLF> or <FF> where I need it

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:30:10 +0100

Am 30.11.2020 um 17:49 schrieb Larry Masinter:
> when I visit
> https://tools.ietf.org/id/draft-masinter-mime-web-info-00.txt
> <https://tools.ietf.org/id/draft-masinter-mime-web-info-00.txt>  with
> three different browsers (Edge, Chrome, Firefox) only one (Firefox)
> without any helpful sniffing/transformation, I suppose I could use curl
> or wget (curl vs Wget (haxx.se)
> <https://daniel.haxx.se/docs/curl-vs-wget.html> (using WSL on Windows? )

Control-S works for me in both Firefox and Chrome (in the latter the
menu entry is nested below something like "other tools").

Also, I don't get how "save as" is different from "save"... I think it's
called that way because you get to choose the filename, that's all.

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 30 Nov 2020 17:33:40 +0000

> [Roman] Thanks for clarifying.  However, I don’t think that consistent HTML+SVG saving is the use case in question.  I believe it was that browsers were not consistently downloading text files.  As…

But I don't have the option.  From my browser (IE) I only have 'Save as 
...' and the 'Save' option is greyed out and this is the behaviour I am 
used to when accessing RFC-Editor, data tracker, IETF etc and I think 
that that is the case for most if not all web sites.  My alternative is 
to select all, copy and paste into a word processor or editor but it is 
not an approach I would commend.

Tom Petch

---

From: Christian Huitema <[email-masked]>
Date: Mon, 30 Nov 2020 09:32:32 -0800

On 11/30/2020 9:07 AM, Theodore Y. Ts'o wrote:

> who don't want to deal with any kind of change, because while*this*
> change might be relatively easy to work around, the*next*  one might

I would refrain from assuming what motivates people -- not inside their 
brains and all that. Some of that is probably a form of conservatism, 
but I also see something else, a reluctance to "rely on the web" and a 
desire for simple tools. FTP is pretty uncomplicated: ask for a file, 
get a file (at least in binary mode). The web on the other end mixes 
lots of stuff together -- page composition and formatting, use or 
avoidance of surveillance tools, detection and specific handling of 
different MIME types, etc. The web constantly evolves. The evolution is 
driven by a mix of usability concerns, business concerns, security 
requirements, and also the need to visualize the pages and support 
interactions on a variety of platforms. We may debate whether FTP is the 
right tool today, but we should acknowledge that they might be a role 
for simple tools that do just one thing and do it well.

-- Christian Huitema

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:41:59 +0100

> when prompted by Julian's post but tomorrow, well, tomorrow is another day!

On what URL?

Best regards, Julian

---

From: Roman Danyliw <[email-masked]>
Date: Mon, 30 Nov 2020 17:43:23 +0000

Hi Tom!

> Sent: Monday, November 30, 2020 12:34 PM
> To: Roman Danyliw <[email-masked]>; Larry Masinter <[email-masked]>
> Cc: [email-masked]; Ned Freed <[email-masked]>;
> Keith Moore <[email-masked]>; [email-masked]
> > Cc: Ned Freed <[email-masked]>; [email-masked];
> > Keith Moore <[email-masked]>; [email-masked]
> > I provided a link to MIME Sniffing Standard
> > (whatwg.org)<https://mimesniff.spec.whatwg.org/>
> work (rather than multipart/related) if they are successful, and if the browsers
> decide to adopt it.
> >
> > [Roman] Thanks for clarifying.  However, I don’t think that consistent
> HTML+SVG saving is the use case in question.  I believe it was that browsers
> were not consistently downloading text files.  As to the “save” vs. “save as”, I
> won’t presuppose the workflow of users.
> 
> But I don't have the option.  From my browser (IE) I only have 'Save as ...' and
> the 'Save' option is greyed out and this is the behaviour I am used to when
> accessing RFC-Editor, data tracker, IETF etc and I think that that is the case for
> most if not all web sites.  My alternative is to select all, copy and paste into a
> word processor or editor but it is not an approach I would commend.

Can you try Control-S or F10-File-Save-As, these are my options on IE 11.  Screen cap attached.

Regards,
Roman

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:47:09 +0100

> unwelcome and is what I referred to as broken.

Apologies, read this too late.

I just tried, and it works as it should for me.

> I see no sign of Text/Plain (which, as you gather, is close to my heart:-)

Are you 100% sure you're inspecting the correct file?

Best regards, Julian

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 30 Nov 2020 18:51:21 +0100

> On 30. Nov 2020, at 18:47, Julian Reschke <[email-masked]> wrote:
> 
Are you using the same browser as Tom (on the same OS)?

Best regards
Michael

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 18:55:19 +0100

> Are you using the same browser as Tom (on the same OS)?

IE 11.something on Windows 10.

Best regards, Julian

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2020 12:59:20 -0500

>inserting <CRLF> or <FF> where I need it

In this case, you are indeed wrong. HTTP returns whatever it returns
other than some options for compression and decompression that don't
matter here. All of the display problems you are seeing are due to the
inconsistent ways that browsers interpret the text files they fetch,
which is not surprising since browsers are designed to display
formatted HTML, not unformatted text. This tells us that web browsers
are a poor tool to look at text files, not that anything is wrong
about HTTP.

If you use a batch tool like wget or curl to fetch the text version of
an RFC (or any other text file), and look at it in a text editor, it
works fine. For that matter, if you use a browser to download a file
using a right click and a 'Save Link As' option, that works too.

Since web browsers are designed to display HTML we also provide HTML
versions of RFCs designed to render well in web browsers.  But you have
to pick the right version for the right purpose.

R's,
John

---

From: Jared Mauch <[email-masked]>
Date: Mon, 30 Nov 2020 13:03:16 -0500

> On Nov 30, 2020, at 12:59 PM, John Levine <[email-masked]> wrote:
> 

One of the questions I always have is how do you re-bootstrap things assuming
Something really bad happens.  

The shift to protocols that a user can’t speak over telnet/netcat really bug me
in this regard.  Yes, I can openssl s_connect assuming I have that on my device
and the TLS police haven’t turned them all off because they’re too weak for
time(&now).

This is why the discussion really bugs me more than anything else.

- Jared

---

From: [email-masked]
Date: Mon, 30 Nov 2020 08:03:18 -0800 (PST)

> I'm sorry, I still don't following.  Clicking on what link did you get
> HTML-ized text and then text another time?

First, the point you appear to be missing is the time frames involved. We're
talking about months or years in the past and stability for years in the
future, not what happened this morning. In that regard, nobody is going to be
able to tell what link they used. Indeed, part of the point here is that people
stop using and then quickly forget about stuff that doesn't work rather than
trying to get it fixed.

Second, why would you assume that behavioral changes have anything to do with
the URL or associated source file you provide?

> If one goes to the datatracker (say https://datatracker.ietf.org/doc/rfc7230/)
> and one clicks "plain text", is one not getting plain text?  If one visits,
> https://www.rfc-editor.org/rfc/rfc7230.txt, is a TXT not returned?

Others have responded to this; I see no need to elaborate further.

> If one is clicking on txt, html, pdf, htmlize, or with errata (all of the
> flavors from the datatracker) and not getting txt, html, pdf, html and html,
> respectively, this is a bug.  Let's troubleshoot it when it happens.

If I attempted to troubleshoot every link or download problem I encounter in
daily web use I would have no time to do anything else. And I don't mean to be
rude, but as far as priorities go, as things stand spending lots of time
helping the IETF fix their web stuff is not even on my list, let alone anywhere
near the top.

Indeed, immediately prior to writing this response I had a little tussle with
DocuSign where I didn't get to a document fast enough and had to request an
updated link, which I mistakenly did twice, which earned me two links, one
obsoleting the other, that then arrived in the wrong order.

I had to resolve this one because it's job-related and there's no
alterantive service. Beyond that, sure, I could file a problem report with
DocuSign, but I think the problem may have more to do with the URL rewriting
crap my employer insists on using. Unwinding the whole sequence to figure out
the root cause? I'd rather have my teeth cleaned.

				Ned

---

From: Greg Shepherd <[email-masked]>
Date: Mon, 30 Nov 2020 10:12:51 -0800

> This is why the discussion really bugs me more than anything else.

Wise words from a man whom no doubt has painted himself into a corner a
time or two. :)

- Shep

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 13:30:12 -0500

> Changed once or twice over years or decades is not "constant flux".

The aggregate of changes to many different interfaces or services or 
tools, each changing once or twice over a few years, may look a lot like 
constant flux.

(what's the saying?  "No raindrop thinks it is to blame for the flood.")

> have to adjust their workflow every few years.

Something I've consistently found throughout my career is that people 
don't like to change from tools and interfaces that work well for them, 
to new tools that require different habits of working.   For example, I 
knew professors who clung to VMS MAIL for years even though it was never 
a great tool for managing Internet mail and even gmail's web interface 
(as much as I personally loathe it) was arguably much better.  I first 
got James Gosling's emacs to run on VMS in 1980, and I've been using the 
same key bindings ever since.   I've adapted more editors than I can 
count to use those key bindings.   I vastly prefer aircraft instrument 
panels with "steam gauges" to modern instruments that try to cram all 
information into one tiny display and are also more difficult to read 
(the old instruments were very carefully designed to maximize 
readability even in low light conditions, part of which meant they were 
relatively uncluttered).   etc.

IMO, that reluctance to change interfaces is entirely understandable.   
People literally invest years or decades into learning to work 
effectively with a certain set of tools, and trying to use new tools 
sometimes drastically affects those people's ability to work 
effectively.   (And sometimes the people get blamed for this.)

Basically I think there needs to be very good reasons to force people to 
abandon tools and interfaces that work well for them. And it seems that 
there's a very unfortunate and widely-held belief that "newer is better" 
that simply is not true in the general case (e.g. does not withstand 
rigorous measurement).

> IETF participants' work are added, with no means of transitioning off

There is indeed a problem here, but part of the problem is a constant 
demand for "new features".   It's not clear to me where that demand is 
coming from.   It seems to me that it's often better to design stable 
interfaces that won't need much change over time (because people don't 
like to change user interfaces) than to keep changing things.   Every 
time Windows changes versions and user interfaces, I hear my 
Windows-using colleagues gripe about how inconvenient the changes are.   
The other day I needed to format a USB stick exactly as Windows would do 
so, so I asked a friend if I could use her Windows machine to do that.  
It took the two of us about two hours to figure out how to do it, 
because none of the old user interfaces worked and the user interfaces 
that did exist had changed all of the terminology to the point that we 
couldn't tell for sure exactly what the PC was going to do.   And all I 
needed was to initialize a USB stick with a single partition and a 
single Windows file system on that partition.

If I look at old automobiles there are lots of different ways of 
operating them.   But fairly soon in the development of automobiles, a 
fairly common user interface seemed to emerge.   I don't know of any 
modern car that is steered with a lever.   Even though newer automobiles 
have some newer features the basic driving interface has mostly stayed 
compatible since the 1930s or so.   Maybe we're just still in the early 
phase of Internet user interfaces and things haven't settled out yet.

> doing so for philosophical reasons, more than anything else.

I don't think it's "philosophical" to want to have stable and effective 
interfaces.  I think it's human nature.

But the fundamental nature of engineering is taking components with 
known and predictable characteristics (because they are designed, 
tested, and/or selected to have those characteristics), and assembling 
reliable systems out of those predictable components.  Without 
predictable components, the whole discipline of engineering falls apart 
and becomes basically guesswork.   So a lot of us understand at a very 
deep level that when you start trying to use components that don't have 
predictable behavior, things break.

 From that point of view valuing predictable and stable services is not 
philosophy, it's reality.

And this kind of breakage seems to be happening with increasing 
frequency.   For example, it used to be that applications could count on 
the network making a best effort to deliver packets intact from source 
to destination.  Since the network was making a best-effort, there was 
no need to second-guess the network. Nowdays, applications cannot depend 
on that happening.   There are middleboxes in the network that try to 
second-guess the applications (whether to "improve" performance or to 
enforce restrictions or whatever)   And then the applications have to 
second-guess the network and try to work around the damage. That's a 
classic tussle which we're all familiar with.  No matter how it's 
resolved (if it even is), does not promote the development of reliable, 
predictable systems.

> For

Unless, perhaps, you're using one of those new very thin notebooks with 
a small amount of flash memory, or you're trying to work from your phone 
or tablet.   Just because my last N laptops have had at least 1TB drives 
in them, doesn't mean the next one will.   (Apple wants a LOT of money 
for an M1 laptop with a 1TB drive.)

But I think this is missing the point.   It's not just that people may 
need to change sources or protocols - though in the aggregate that is a 
problem - but also that the new protocols being proposed and interfaces 
presumed are less functional than the old ones in important ways, and 
their behavior is also less predictable.

If IETF said, for example, we're going to change from FTP to WebDAV as 
the means to provide remote file access to our documents, some of us 
would adapt.   We'd write new tools or modify existing tools if we had 
to.   The same would be true if IETF decided to use anonymous NFS, or 
anonymous CIFS, or sshfs, or FTPS, or whatever.   But what we were 
essentially told is that we'd have to completely do without such an 
interface and (at least in the initial message) that the decision had 
already been made (it was in the past tense), and that the powers that 
be had decided this based on extremely flawed analysis and basically a 
disregard for any use of the existing FTP server other than mirroring.

Note also that /any/ change at all will still probably deter some users 
to the point that they stopped participating in IETF or reduced their 
participation significantly.   That's part of the cost of changing 
interfaces, and needs to be considered as such.   And unlike some, I 
think that people who have habituated to familiar workflows and 
interfaces should not be disregarded out-of-hand as if they were irrelevant.

More broadly,  anyone who thinks that they can predict how other people 
/should/ behave under changing conditions is likely to be surprised, 
even moreso when they think they have a right to demand that people 
adapt as they expect.

> portals and other technologies should get retired is still going to

Well, every change is an unknown.   You don't know how much trouble it's 
going to be, but potentially any change might be a lot of trouble and 
very disruptive to other things you need to do.   For that reason alone 
there's a tendency to avoid such changes.

As for the long-term question:   IMO the thing to do is to very 
consciously pick interfaces (machine and user) that are standardized, 
functional, and can be stable over long periods of time, so that there's 
less frequently a need to consider such questions.   And sure, once in a 
great while there's an unavoidable need for change, and when that 
happens, those transitions need to be managed, generally with plenty of 
advance notice and overlap.

> older mechanisms forever.

Certainly I think a transition window is often better than a hard cutoff.

Keith

(I still suspect it might make more sense overall to fix FTP than to 
abandon it.   But part of that is based on a realization that FTP is 
still the tool of choice for some circumstances and usage scenarios that 
needs to be maintained; and also a belief that IETF should eat its own 
dog food whenever it makes sense.  But it might also be the case that 
say WebDAV is better for IETF's document access purposes even if FTP 
continues to be useful elsewhere, and dog food consumption is not the 
most important thing in this case.   I just don't think the dog food 
consumption should be disregarded out-of-hand - it does affect both our 
reputation and the quality of our work. )

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 13:35:42 -0500

>> you want, vanilla HTTP doesn't provide what a tool needs to reliably do

No, it does not.   Vanilla HTTP doesn't have a way of listing what files 
are available or what their characteristics are. WebDAV PROPFIND can do 
that, not vanilla HTTP.  The content you get back may have links to some 
files but you can't reliably distinguish from links that are files in a 
collection like RFCs or I-Ds, and links that are pointers to other 
things.  Sure, you can make a tool that tries to screen scrape whatever 
it gets back and make some sense of it, but there's no reason to believe 
that such a tool will be reliable or continue to be reliable.

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 13:39:48 -0500

On 11/30/20 12:33 PM, tom petch wrote:

> But I don't have the option.  From my browser (IE) I only have 'Save 
> as ...' and the 'Save' option is greyed out and this is the behaviour 
> I am used to when accessing RFC-Editor, data tracker, IETF etc and I 
> think that that is the case for most if not all web sites.  My 
> alternative is to select all, copy and paste into a word processor or 
> editor but it is not an approach I would commend. 

When downloading a file via FTP, the client knows what the file name is 
on the server, and it's often reasonable to save the file on the client 
machine with the same name.

When downloading a file via HTTP, the URL might be anything at all, and 
there's no reason an HTTP client should assume you want to use that 
filename on your client.   So in practice an HTTP client should 
generally expect you to specify the destination filename as default 
behavior.

Keith

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 19:50:57 +0100

> generally expect you to specify the destination filename as default

...but then it will usually (absent content-disposition) default to the
last path segmemt, which would be the right thing for the URLs we're
currently discussing.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 13:52:49 -0500

> last path segmemt, which would be the right thing for the URLs we're

Right, but it explains why web browsers may offer "Save As..." (if you 
can find it) but not "Save".

Keith

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Nov 2020 19:54:52 +0100

> that such a tool will be reliable or continue to be reliable.

I read "if what actually need to do" as applying to a human user. And
yes, in that case you usally do get some sort of UI that let's you pick
the files you want. It's just not a standardized UI, as it depends on
what the web site developer supplies/configures.

What you apparently want is something that can be used programatically.
One candidate for that would indeed be WebDAV.

Best regards, juliaj

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Nov 2020 14:23:03 -0500

> what the web site developer supplies/configures.
And you have to specify each file individually, and then specify the 
destination filename, and check back to see that all the files you 
wanted get downloaded successfully.
> What you apparently want is something that can be used programatically.

Yes, because that makes it possible for variety of tools each intended 
for different use cases to each work reliably.

Keith

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Nov 2020 14:37:07 -0500

One micro-addition to John's comment without disagreeing with
anything he said...

--On Monday, November 30, 2020 12:59 -0500 John Levine
<[email-masked]> wrote:

>> As I just posted to Carsten, this was a major problem 35
>> years ago which  is why protocols like FTP are rich in
>> options to handle this.  I could  be 180 degrees wrong that
>> HTTP/HTML is making no changes and so giving  just what is on
>> the web site which is then no use to me and it is FTP  that
>> knows what I really want and is making corrections for me,
>> such as  inserting <CRLF> or <FF> where I need it
> 
> In this case, you are indeed wrong. HTTP returns whatever it
> returns other than some options for compression and
> decompression that don't matter here. All of the display
> problems you are seeing are due to the inconsistent ways that
> browsers interpret the text files they fetch, which is not
> surprising since browsers are designed to display formatted
> HTML, not unformatted text. This tells us that web browsers
> are a poor tool to look at text files, not that anything is
> wrong about HTTP.
>...

You are also a tad wrong about FTP.  If FTP is run using the
ASCII Type and "stream" mode (which are the required defaults)
and if no type of structure commands are issued, then a
conforming server will deliver an NVT text file (see the Telnet
protocol spec) using as line delimiters.  If the file is coming
from a system that uses CRLF, character counts, or record
delimiters internally, "raw" CR or LF should be transmitted
unchanged, but, if it uses some other line-delimiting
convention, the server it is required to turn that into CRLF.
Presumably if the local system using page delimiters other than
FF those would have them converted too.   But FTP won't do
pagination of an unpaginated file or introduce line endings into
a file that does not have them.

There is also a small, but unpleasant, elephant in that room.  I
have seen many FTP clients, most operating in a command-line
mode or with URLs, that will not allow the user to issue most
(or any) of FTP's large selection of commands.  For text files,
that should be ok because the defaults are supposed to be TYPE
ASCII, Stream mode, and File structure.  And, while I have never
taken the time to track down the details, either some of those
clients quietly request TYPE Image or some servers transmit
whatever they have as if Image files had been requested (I think
the latter).  That works much of the time, but can result, e.g.,
in LF-delimited lines in what should have been ASCII transfers.

And, looking back at the spec to refresh my memory, I
rediscovered a glitch I found and tried to get fixed years ago
but got nowhere.  Watch for a note on the subject.

   john

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Mon, 30 Nov 2020 22:07:18 +0000

It reminds me when (late 80's) I first tried to install NCSA/Mosaic web server and browser from source code... *ALL* the documentation was written in weird .html files... (and this predates commercial Internet in Europe, the tarball was received over UUCP)... Took me a while to simply open them with emacs + writing some C code to remove those 'tags' ;-)

-éric
-----Original Message-----
From: ietf <[email-masked]> on behalf of Jared Mauch <[email-masked]>
Date: Monday, 30 November 2020 at 19:04
To: John Levine <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Bootstrapping the internet (Re: When all you have is a hammer, everything looks like a thumb, or Call for Community Feedback: Retiring IETF FTP Service)

    - Jared

---

From: Adam Roach <[email-masked]>
Date: Mon, 30 Nov 2020 18:42:40 -0600

On 11/30/20 11:33, tom petch wrote:
> But I don't have the option.  From my browser (IE)...

FWIW, Microsoft is ending support for IE in August. You won't get any 
security updates after that date. It would be *highly* inadvisable to 
use IE on the public Internet in any form after August 17th.

/a