I have reviewed this document as part of the operations directorate's ongoing effort to review all IETF documents being processed by the IESG. These comments were written primarily for the benefit of the operations area directors. Document editors and WG chairs should treat these comments just like any other last call comments. Summary: This draft is on the right track but has open issues, described in the review. I apologize for this review showing up a couple of days after the end of IETF Last Call. This draft discusses considerations for what are effectively MPLS-in-MPLS tunnels for multipathing - an LSP carries another LSP, with the inner (client) LSPs being distributed across multiple outer (server) LSPs that use different paths. This draft is motivated by considerations specific to MPLS-TP that complicate multipathing when one of the two MPLS instances is MPLS-TP. I consider the following to be open issues that need attention (each letter is used to tag the issue in the text that follows - [B] occurs multiple times, as there are multiple aspect to that issue): [A] Summary text in Introduction appears to be wrong. [B] Section 3.2 needs a summary. [C] Entropy label "sandwich" seems peculiar and at least needs an explanation. [D] Discussion of multipath impacts on MPLS OAM should be added to Section 4. --- Comments on draft text [A] Section 1 (Introduction) 2nd paragraph: RFC 5654 requirement 33 requires the capability to carry a client MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654]. This is possible in all cases with one exception. When an MPLS LSP exceeds the capacity of any single component link it MAY be carried by a network using multipath techniques, but MAY NOT be carried by a single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation imposed by MPLS-TP Operations, Administration, and Maintenance (OAM) fate sharing constraints and MPLS-TP LM OAM packet ordering constraints (see Section 3.1). Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above exception appears to concern the reverse, and Section 4 on carrying MPLS over MPLS-TP allows for MPLS LSPs that exceed component link capacity. Was the exception intended to be for an MPLS-TP LSP that exceeds the capacity of any single component link? Also, "MAY NOT" is not an RFC 2119 term (for good reason - it's equivalent to "MAY OR MAY NOT"). I think "cannot" was intended her, but "MUST NOT" is another possibility. [B] Section 3.2 contains a lot of useful details, but it could really use a summary at the end on the options and recommendations for meeting the four requirements (this summary could be a new Section 3.3). It's a bit difficult to discern what's going on, as I had to read this section more than once in order to discern the following: [C] In Section 3.2, use of an MPLS entropy label to meet requirement MP#1 requires that it be "below the MPLS-TP label" whereas, to meet requirement MP#2, the MPLS entropy label needs to be "just above the MPLS-TP LSP label". This combination appears to suggest "sandwiching" the MPLS-TP label between two entropy labels that effectively carry the same information (albeit, applied and removed at different places in the network) - was that intended? This is important, as there seems to be no reasonable alternative to the entropy label for meeting requirement MP#2, and the draft's alternative to the entropy label for meeting requirement MP#1 is "some form of configuration" - the entropy label would seem to be preferred to that based on my reading of this draft and the comment in RFC 5706 section 2.2 that "Anything that can be configured can be misconfigured." [B] A summary of requirements for implementation and deployment at the end of Section 3.2 would have made this much more obvious. That should be provided in addition to addressing the above question about duplicate use of the MPLS entropy label. --- Relevant Q&A based on RFC 5706 Appendix A I'm omitting management concerns, as both MPLS and MPLS-TP have extensive management infrastructure, and this draft's notion of stacking LSPs so that one instance of MPLS (or MPLS-TP) can be a server to (i.e., carry) another is not novel to this draft. A.1.1. Has deployment been discussed? [B] Yes, although Section 3.2 is weak on deployment requirements (some of which are implementation requirements) and could use a summary. A.1.2. Has installation and initial setup been discussed? No, although because this draft reuses existing functionality in different configurations, this really reduces to the A.1.9 configuration question below. In practice, quite a bit of operator knowledge and insight is required to get initial setup right, e.g., as traffic engineering LSPs to avoid congestion problems (needed to meet requirement MP#3) requires significant knowledge of network structure and expected traffic flows. This should be obvious to anyone who works w/MPLS, but it's probably worth stating for completeness. A.1.6. Have suggestions for verifying correct operation been discussed? Yes and no. Section 3.2 discusses OAM, and contains this important requirement for OAM traffic: An Entropy Label must be used to insure that all of the MPLS-TP payload and OAM traffic are carried on the same component, except during very infrequent transitions due to load balancing. [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in Section 4; that should be added. A.1.9. Is configuration discussed? [B] Not much. Configuration is required to meet requirements MP#3 and MP#4, and is one of the options for MP#1. A summary of what has to be configured as part of the summary at the end of Section 3.2 would be a good idea. A.2.* [skipped, see above] A.3 Documentation There's quite a bit of operational material contained in this draft; a separate section on operations and management considerations would not be appropriate. There is a useful implementation status section which appears to imply that MPLS-TP over MPLS is not currently deployable due to absence of entropy label support, and because deployed MPLS equipment does not generally support multipathing. Nonetheless, data center experience indicates significant value and widespread use of multipathing based on link aggregation and ECMP; corresponding benefits can be expected for use of multipathing in the Internet beyond data centers. --- Editorial comments/nits: - Section 3.1, after RFC 5960 quotes: [RFC5960] paragraph 3 requires that packets within a specific ordered Insert "Section 3.1.1., before "paragraph 3" and likewise for "paragraph 6" later in the same paragraph in the draft. - Section 3.2, requirement MP#3: MP#3 It SHOULD be possible to insure that MPLS-TP LSPs will not be "insure" -> "ensure" or "assure". Surely this draft does not envision operators taking out insurance policies on MPLS TP LSP behavior ;-). - Section 3.2, last paragraph on p.8: An MPLS-TP LSP may not traverse multipath links on the path where MPLS-TP forwarding requirements cannot be met. "may not" is meaningless - I believe "MUST NOT" was intended here. - Section 4, middle of 3rd para: Editorial suggestions for clarity: OLD For those LSP that are larger than component link capacity, their capacity are not integer multiples of convenient capacity increments such as 10Gb/s. NEW For those LSPs that are larger than a component link capacity, the LSP capacities need not be (and often are not) integer multiples of convenient capacity increments such as 10Gb/s. In particular, "are not" seems wrong, as the "integer multiples" case is possible, so I changed "are not" to "need not be (and often are not)" in the suggested new text. Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA  01748 +1 (508) 293-7953             FAX: +1 (508) 293-7786 david.black at emc.com        Mobile: +1 (978) 394-7754 ----------------------------------------------------