Dear Authors,

thank you for bringing your proposal to = the discussion. I agree with your view that the explicit routing enabled by= SRv6 creates an environment where PREOF can be used. And, as we know, The = PREOF=C2=A0may be used in a DetNet domain to lower packet loss ratio and pr= ovide bounded latency.=C2=A0

After reading the draft, I've go= t a question for you. What do you see=C2=A0as the difference between the IP= v6 Flow Label per RFC 6437 and the Flow Identification field in the TLV pro= posed=C2=A0in the draft? Could the IPv6 Flow Label be used to identify the = flow for the PREOF?

Regards,

Greg

<= /div> --000000000000fb666005bce3edb9-- From nobody Sat Mar 6 20:35:39 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A40C3A1E11; Sat, 6 Mar 2021 20:35:37 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.8 X-Spam-Level: X-Spam-Status: No, score=-1.8 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XA1UsI70LTP0; Sat, 6 Mar 2021 20:35:34 -0800 (PST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70057.outbound.protection.outlook.com [40.107.7.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BEA4C3A1E10; Sat, 6 Mar 2021 20:35:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X+u/SCsk5NlRlcQDUyZzdy2nc5MbnlVBDP8ji4lMuNzKnTmqm502mSWaY7K5v/+YR2igsEaH51lpaMXKYJVThGnoXfAJ0F+5UZqtUv4a/RLtcZUFIBti1qJbnKtM6YEKggbb8EL9zLa7Nawmz+xKxz5GOdMO4nYMCTaJQsDSPQ+jJ2FmsnqO+rbBXWLScUSEXrNZ/vcYMoFq+wwPaKMeaTLNIe3GoE2ZDNr+xrVVjgCUIUgTqwL77LytKGlQsMsF6oKwLjWz898OgK2c5OCAzJNlwbDkaip8pSoLar6CoIhOd6XOC3MRHB1iVTqmq5ElUhk35OydovnWPtVrR+H0ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F/tdBiRJ2wpMl0uOtJQvmORb10M5ZgDTgp5dBAyAotc=; b=YjGXwn1enEk8dgknD7GXjjOI2lNhg6LJ4A7Bfg4t59SEzsRUSFiyjZ1u/R7VBJkd1iDLRzYA6RQxECe+ghqRi3Ooa9DQPhXFX6u3NhBPfBPqCOiZcQqFbctbnSM6hsQ7mcwYi7PV/H2TSjNzGgwINk1qaM4f5THMqTF60vzmoOZIGeKduLh9guB33RUd6I0OxfpJCv33hJZ3vJPMfVeg5koJr8cQHHz0xsnsRbYP+STMjgq8/i84Bj1xK6TosGltUxpEge9ar0lfICi7cm539O1qWHhuxRYkbzIabJWbrWaYPalJpkCuWJr39r6czOU2ng6QjsMBxna3Q0Mx2Rj+RQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F/tdBiRJ2wpMl0uOtJQvmORb10M5ZgDTgp5dBAyAotc=; b=Du3VY2tfsE0aZS8xDgsWoaVbZZZnlCVHTMEWGowIsYRhOcyXH5f3oWTlYbYvqcF7taWGjOZQi+f0WtxOMQi7y8H7ChgkjRieCO2YReC8XAC6UmMGjFOThfXWVdM13n5uVtDeb6XCpfHb5hagu76P8gzn6XFmnzUHd+iCpsqBlgE= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM9PR03MB7171.eurprd03.prod.outlook.com (2603:10a6:20b:263::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Sun, 7 Mar 2021 04:35:31 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.022; Sun, 7 Mar 2021 04:35:30 +0000 From: Yaakov Stein To: Haoyu Song , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTA= Date: Sun, 7 Mar 2021 04:35:30 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: futurewei.com; dkim=none (message not signed) header.d=none;futurewei.com; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: f463696a-fea4-4e43-bc07-08d8e12270b8 x-ms-traffictypediagnostic: AM9PR03MB7171: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3qAszMY9fUA8ZCctC3QRi7b/nz4WenSCoh132gQ+3XPwgmC2ltb49nQhWGcoyJbjnYduPOmQBpx+V8BbIeUf5kxDld1UscVZqYlydNCNPvrYg3ntaqmVG1H5PoBaO5NtX63sX6jXvS81+vqn7O7gvI/mBiydjmuQ/7uhKmPCfe90wQSM3NpGEu8lgEOV1kstX7Vl74ifS2BQ9VOnd7yc3DG4AsLfHY//l9y6VDTmiJ7J7fej/4HikUnN5qQ3La6P9aVPjAun9TKONwnqVH2ZoKOrWy9NhflhJzXtKWLpCOHPm1ncgsC+I/67IN7w9n8UzdAAPiWlgbn9yNmZNT0AOsr7MOeYGa94R6YyMmSZcSZ3Hz7pkmR+yU9OceshFrFAPq9k1OjEdT7HcPsVOTnOomPX/7gsxXZ48YGE1JajKumEAWpddil5QrQxrvtgjjxwJ5/h2du2OVP/hBg9ZjOPu8lXcktSkBisBikE1cPFUo7IlftPJ2aaTYgVl6DTx9BCMs6lP4JGR7J1DCGUN1Zp8PRUg80eCnYtQGO+sdvu6NxiobdnihN4eOvzKiAYOek9qXVD6CYZPaIcW9XbTmdRMhj9s7CRWrdYvV28JSARvJg= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(366004)(39850400004)(346002)(396003)(136003)(7696005)(26005)(33656002)(2906002)(55016002)(9686003)(110136005)(186003)(66574015)(8676002)(66946007)(316002)(5660300002)(966005)(66476007)(52536014)(86362001)(66446008)(66556008)(64756008)(6506007)(53546011)(71200400001)(8936002)(478600001)(166002)(76116006)(83380400001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?c/XgLesKRDExaBsYCIjDXlR+Lnh5dlNzGcHw/DQSlOQAds1OnYvn2rJxgK0x?= =?us-ascii?Q?/QaJhk+PUPIMSN65ygz2Ss8pwEO7aggADSKeBfays2+2fgZFrXWbZugYAbcY?= =?us-ascii?Q?1rObAXHcJ4kUIw4cQynxrMifIdn0mjesHkq2fNuyrVNlI4EltGFO6K160C7L?= =?us-ascii?Q?MDOS0w2m5THElNdzta5B+0HwC3pmDEhkHadFow2/LS3btfD3J46hCx9QbUB/?= =?us-ascii?Q?ZrNCUaUONNrwmMU8/1PTNTVDz7HzyVHNTSyUEbg+EZMX/ucnNwxSQPfX7iB7?= =?us-ascii?Q?j00Cjuw/RoJWdOb0sUMq1ex0zJwgBRSpplqqzrJdPRIAsEQxsHeSwOAsqCi/?= =?us-ascii?Q?pdTciJ+Ef56Htx0JnnLq+gTVnhdvKnqIzr9jALVmaioNmBOk0V71hpyUTAY8?= =?us-ascii?Q?nm7iuN9dxv4VBdWWCYeWfMUycMS9Ek3pPRO/NrYPtNBa5HWjsR/9bpPk11vX?= =?us-ascii?Q?l+r12Cjn3dViHuIXSHd1965uwzS81DO7QjTGoot+YwE3rjk0lOag/+RxnQ0M?= =?us-ascii?Q?asR7hCSw14zdiKuW7JhQ19sY3FCotYQIzsQr3OOPcpfUODVzMnw6Ed7827Qk?= =?us-ascii?Q?PeWsA7uCWRU0JUYD3LVxd3yrwssq7LJx0L/kIqUBjLJm71KZ6yOXFJel7Gmv?= =?us-ascii?Q?N9MhcN7TghZvojBh9MsfrGu6H89vtmpsLoWC1Wr2Mjm7tnhnHJEtV94/L1Zn?= =?us-ascii?Q?BFdd1d2MSCTiv7ZWfSG1tabq1FGUVUJt1ecWDMFDPLaa4yPlSOMoGkhIU5PG?= =?us-ascii?Q?V/DtD6We+vpkieNq/vVOxQPCYLNWWJJohyowX+QGNe/mQ653twBNQT4gRnsu?= =?us-ascii?Q?eUeV2Ag4qzxlfomkUcnou3vx76VmKqRZWrvJAIPUgUkXOLlKrFTIgIC5nBfP?= =?us-ascii?Q?ispn9Gtrc92Fc2tjbCcBHiw4RWmyx4utjCRiGY5AatJf8U4fnAGloqhn7EWW?= =?us-ascii?Q?szTWYmxg2eTeGIXZZbOVCkNkF3soqcRK5GRGk9lwbpXzr78WzrC5HwrdJuwT?= =?us-ascii?Q?lgozS+zhKdFCX7pl3x2+y6FWb7gqY7FfdsDXo+F1yNoFI/lDky1B/ag5wnfS?= =?us-ascii?Q?cEEmwVQ2wS646M9HYujvYkADpbOizymjSDFR7dRcxeRDTNOt+ugyfQfXkKO+?= =?us-ascii?Q?RXT8j88DoIrh7wEhmr/4iQLjgBc3/vsZIl5qG35HRu4qVnXypee4InOm12MR?= =?us-ascii?Q?Q/0iLOuDNDJOAdeYULt5afSHIBqDjDFbAJrXkL545rgQ/+AgA04V/SHJd1xm?= =?us-ascii?Q?1uPH1rRinJumVAftQ2lE1dkDYJEesSzGx+u/4/UtWg74n4eAj2It9xDykLOQ?= =?us-ascii?Q?O+PgshRVKAkaQv5XdmxN66S2?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR03MB3522BD9D4D0A3134FE16B49FE5949AM0PR03MB3522eurp_" MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f463696a-fea4-4e43-bc07-08d8e12270b8 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Mar 2021 04:35:30.8438 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4kbgzVSASN01+R24/yMsIz1vvJ/4FvhZ6ak+CnppRmAxaBwhLsC92VlfezugFhHay6Ovb/WFZqka5y/BjxSJKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7171 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2021 04:35:37 -0000 --_000_AM0PR03MB3522BD9D4D0A3134FE16B49FE5949AM0PR03MB3522eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. Y(J)S From: Haoyu Song Sent: 05/03/2021 22:46 To: Yaakov Stein ; detnet@ietf.org; spring@ietf.org; pce@= ietf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR03MB3522BD9D4D0A3134FE16B49FE5949AM0PR03MB3522eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Haoyu,

I’ll address your points:

> The use of clock time as deadline requires netw= ork synchronization

That is a basic assumption of TSN networks (IMO the = defining assumption).

However, the sync needn’t be highly accurate &= #8211; it depends on the tightness of the delay budgets.

> accurate measurement of per-link propagation ti= me

Yes, I am assuming that once an for all someone does= a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measurement of the links, a= nd these are stored in some database.

Once again, the required accuracy depends on the del= ay budgets.

> which can somehow limit the application scope o= f this work

If the delay budget only above the physically minima= l delay by say less than 100 microseconds,

I agree that the previous two issues MUST be carried= out. But in such cases there is no alternative.

If the delay budget is much higher than that, then o= ne could use an RSVP-like mechanism,

sending a packet (or several packets) from source to= destination collecting a stack of timestamps,

and then using that stack for the following packets.=

> Mechanism should be provisioned to track where = the timing requirement is violated and by how much

I’ll leave the OAM for later. However there ar= e already many high accuracy performance measurement techniques and protoco= ls for this.

> Recently programmable scheduler research has pr= oposed several primitives

Yes, I tried to stress that this ID is not limited t= o EDF (although sometimes that is a good strategy).

One can even reproduce Qbv behavior using a stack of= deadlines (although why would one wish to do so?).

> such as PIPO and PIEO

I’ve heard of PIFO (Push In First out) but not= PIPO. Is this a typo or something new?

I agree that there are mechanisms that are optimized= for hardware, but I have come up with a very nice hardware implementation = for PEDF

and prefer to find hardware implementations for opti= mal schedulers, rather than to determine schedulers based on optimal hardwa= re.

Y(J)S

From: Haoyu Song <haoyu.song@futurewei.com= >
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@rad.com>; detnet@ietf.org; spring@i= etf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Just got a chance to read your draft. I agree with t= he comments of the others that this is a very interesting work. I’ll = just add a few points.

The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget a device latency which require a rout= er/switch to obey. In case the overall budget is evenly divided by the hops= , a single parameter is enough. Of course, if one wants to customize the bu= dget on each hop (which might be necessary considering the different capability/capacity of each hop), a st= ack is still needed.
Mechanism should be provisioned to track where the ti= ming requirement is violated and by how much (e.g., using timestamp or flag= ). This would be very useful for troubleshooting.
Rec= ently programmable scheduler research has proposed several primitives such = as PIPO and PIEO and provided feasible hardware implementations. The scheme= proposed in this draft can easily fit into these primitives.

Best regards,

Haoyu

From: spring <spring-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR03MB3522BD9D4D0A3134FE16B49FE5949AM0PR03MB3522eurp_-- From nobody Sun Mar 7 18:41:33 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE7453A2240; Sun, 7 Mar 2021 18:41:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.989 X-Spam-Level: X-Spam-Status: No, score=-1.989 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=futurewei.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LulpL-3_Z5kd; Sun, 7 Mar 2021 18:41:24 -0800 (PST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2119.outbound.protection.outlook.com [40.107.223.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F5813A223C; Sun, 7 Mar 2021 18:41:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kx5IaaSY15rHvK9BwohbUpXyHf6B55Gnm1NifjjjZsJAaazZQdNTr/HMZtAkl0WR0a+TF29fvJv7QEfnTqXYpAWiBdIiI8eDnk2P/7R1nKciCk+o7cN++AH5S4wXcq+JQEdvJhm4Vf621TCR6CpZ0E0eHl+1ZP9tBz5ak3bRCQx/RgEik7+rrUdCg32OvwuAY9XI5jeKw6HEL+vWeBv8/4IN6ih9b1TlwHz2knuuqxIUq6b14uzind7z78MwfEOtwKDkBWpNb0LEbVVDMd5W7kShxGloQraZSKQCk05JlA1U11gFsPmBbzPaVzVpfGdkWLieVEFdWQFUAEp4B4Zs2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d1kATNH2nRW0JZgp2YfobLhfWaJtJQT9WSMfZ51gLcc=; b=eV7i2frTNwfLrlgoVpiwsJ0F6h5ZuHv+f3HsT4W0kOnSROyVB0r2zi2p6xT9HFXRRabaRR4dB03Ob3wDU+C+XC8UidjYHdo7mzyS2OYaGqoV3JUTW1githoTYn0lGxOO0Mm1Y9n0o1xlzKS9O/xCQZoR+aK63TDnKXyPxCzMD5ypDVGy6nlXGAFAAAGJFFfdCWv2tmwHvd2dx1yuh6VNgtgCCzvEn5R6e3w/lDnHG2YrKAXGXseMEJqJWmznMNbUpgT4qGkEQ5TVmdAVDMYA+7grR5HU3aDzr7qEMSYdvovgGe93eOe0/aXLycYKb1JSffbh4bJwuIatMib2K4mpBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=futurewei.com; dmarc=pass action=none header.from=futurewei.com; dkim=pass header.d=futurewei.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Futurewei.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d1kATNH2nRW0JZgp2YfobLhfWaJtJQT9WSMfZ51gLcc=; b=W09BI+1AbwRas8zdA/xniAofULaTdoLTqVL+1DD0I7nloFzmh90kB4Pg/6DTyOtUXaT4qkBg/kQdUjC7kSn+SXiIBV0/Ee/CmUGRl+rSOZU5xdsutlcL6cPhtnjUQccqM7aYOATNW7a+1yP5FhpQLDXLWYltyajvw4lINTMzouo= Received: from DM6PR13MB2762.namprd13.prod.outlook.com (2603:10b6:5:13c::13) by DM6PR13MB3114.namprd13.prod.outlook.com (2603:10b6:5:19e::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.14; Mon, 8 Mar 2021 02:41:18 +0000 Received: from DM6PR13MB2762.namprd13.prod.outlook.com ([fe80::cd7c:ab34:e7f7:203e]) by DM6PR13MB2762.namprd13.prod.outlook.com ([fe80::cd7c:ab34:e7f7:203e%6]) with mapi id 15.20.3912.026; Mon, 8 Mar 2021 02:41:18 +0000 From: Haoyu Song To: Yaakov Stein , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkA== Date: Mon, 8 Mar 2021 02:41:17 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: rad.com; dkim=none (message not signed) header.d=none;rad.com; dmarc=none action=none header.from=futurewei.com; x-originating-ip: [2600:1700:38c4:650:a9b5:3b9:2c14:3108] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3a931c89-b948-496b-a420-08d8e1dba682 x-ms-traffictypediagnostic: DM6PR13MB3114: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PiuNp+DmmgVFIAcko6ITjo1oqRmkPHQkmIQyZMz0KMOEvGGlUu1Qcjm/m/RVvwJnqHfPMq07811uNLFesm11914gpfNKPabTw+oT1+Rsulc8Iosf5poQGqkdOe20mF492tdFBpVTMPRu8L7tNTQi0wX9tLf3qVCILIxFU4uHL1DAKfehwu0bdQVyaLYMta/iUAda7ync3t7bRdQqPtBPlvUU5e/o0EErM1egfNQZThY9hMdWpoVTGaCFO64EsT7DB4mMOOJPaoNBCD9QQc0qN3yEb8E036e5VZ68GB1QyZC8tY5YuW/V93q/2voD4BuPiK5v8D+mm/AzgPqoIBwVr8mw2XGMXFTVBKGdHBQQU9BapiZfhPWbX6TMzxN0lJwEXfMku5BIo7hDgvUCOznm8HLXr9MSHkaTDWf4GY2egkJAYYb9AUt4fnGbhjxm0hfqADhU5dXPSq4fIczZURteACgf9U5yJEk7+KuR8VTKir86Qf3UNaxmrNWTgAFY+kF4cdseprWrouHWLVx52AqJghPppVSmVhJzCRkYUcf3EiMJW7hP5xisHz7VD72NlgWok504jcck/WAEoIqYz8xKPXtJIELfypmTfXijzQqLhy6VylOnhfpS1Gkkpdh6/sjerutYPlSJpFDobjz3DadBFQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR13MB2762.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39830400003)(366004)(346002)(136003)(396003)(376002)(9326002)(186003)(66946007)(66574015)(66446008)(64756008)(66556008)(66476007)(33656002)(2906002)(8936002)(478600001)(53546011)(6506007)(71200400001)(76116006)(83380400001)(7696005)(966005)(166002)(110136005)(9686003)(86362001)(44832011)(52536014)(5660300002)(55016002)(316002)(8676002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?AGlP/AHC1Q+uRxqO1GC/gSM3ya+NJpmBNsNN4YP2PfP16PDdFPY8Xd7VNa3f?= =?us-ascii?Q?Hd+bR13H5AD+KkVIrtSWfQrc6gHWySzQ+1NAt6/CXTjTsdhOTQPxKiSjj9oj?= =?us-ascii?Q?cZ7CIbiG+Ixt3iK6o3OFiV1jH9ZY1AKdAjkDFNCqN2lQb1pVbVKlnAlIevf8?= =?us-ascii?Q?aGCEquvAbsUYnugH87QBW/axr86Y8+UqqcNwFiNZ0ptmWEC9w/tHS4EOG8is?= =?us-ascii?Q?RqCahqXtGdzCUMaZ0vMKwrkV6Wf7c45WiIGgeD6RIuNJ/92HziSiC0ZXYxOv?= =?us-ascii?Q?/yHo0I34auU1BHQrF+rpKRy0oUg1W1RqpXJvlmWH1n/odiHO1PSfxGP/rJ5u?= =?us-ascii?Q?IHxsOXJJm0kETXsB00/EH4giQBlyd5a2KdJvvMRCWw7z6aRhw2KkCMSOFE4Y?= =?us-ascii?Q?QxM6AN/TLqklDzqK8EYmXiQ+Rcl4P6oppUSYZl2wXpzOFqxPVAlAi0q6i4HE?= =?us-ascii?Q?C8LbnlF4VQv+T43NfPMDyhZYIqBQGS5zTUdh5BBVc3+/nEjXS54qtx5hNcEc?= =?us-ascii?Q?aG72o7dIx47DoN3u1JLXACgxWXMMxRGlZcTNZdRIFMPxsThQ6mGHWtHNTd7m?= =?us-ascii?Q?iL7PwD7ZupL21h9jhvqyB2L1OReQNh1MtNvzR2LDh2hXlB/sQSlInHURqnzW?= =?us-ascii?Q?n41yLv7qpSvHntodtniCBzQ/XPKOOPxmrq/DRPGQQzq6J7PUTZhthx0acWIg?= =?us-ascii?Q?yskMEsS7jlGHpPRvOgwPbfPF5FUd5c7v5nT7madO6EfS+HlK8NqsKL4IC9Xr?= =?us-ascii?Q?3U11UF0Adz73XH8OB2j2D1+4TvdValyFuiCoNRFaAgUWcLnriq9aTz4jeC6T?= =?us-ascii?Q?1/TY15vo7x2275BKJ/h0XNes38Ts4dqL7SwwFVTdF7FzzbNeerClpQrjYbCy?= =?us-ascii?Q?D7DLb6bcMGH71peZ5o7PKFfZPBdRNCPg3L8R9NapRIilvuHzAiz58tncBxGV?= =?us-ascii?Q?/hJDVxjxXjTmN4Zz1Xw6Pb65fyTW5Q+ZN016ssyWbUotvRjvoq33Zs4GtU9L?= =?us-ascii?Q?gmA7DPD9g7ZjWVVyvHzxhODl4wmOwIaHltym7GaNUICme17ZXN/MbXeNnSoy?= =?us-ascii?Q?L+s3BCjo6ta2VYH0ur2EbAUPSXuY+TvAYpY66BoAwqgFyaU7BpTZDHUtdhrB?= =?us-ascii?Q?miAVU0w9N6n8A+QPORjWS++yH7T14i+jtcx+uYYbdAxKLHxY5XS/mxzEJuKr?= =?us-ascii?Q?VI/8J27xVFgsJ4172x6fx5YyY/kpURW7LIbas6UXHEkRkZHNmbxwd0x7tSih?= =?us-ascii?Q?ezj8vyLd1iQX7P0tPYijFk6nmtw1Can4F99MTHIYyK7Rdk4Fut0RKbJL9RLC?= =?us-ascii?Q?9wm4uFAHThgzLYjVPh4z2VxfTir2RRRC0xagvjTe3E4ohNacmtzu0OaCercb?= =?us-ascii?Q?uqZXSvfZFZU1xlAEHTFbxHGRmPDd?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_DM6PR13MB27624F07A612BDCF98A8C92F9A939DM6PR13MB2762namp_" MIME-Version: 1.0 X-OriginatorOrg: Futurewei.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR13MB2762.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a931c89-b948-496b-a420-08d8e1dba682 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 02:41:17.9033 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0fee8ff2-a3b2-4018-9c75-3a1d5591fedc X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: YAbyIzU3NO686J+WSQq0qOVCADwMKLPHj32EjXhL3YzysiUGkYYQ5s2cpi+qlFJ3WMOubqSvDN5p63nDdEUXwA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB3114 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 02:41:27 -0000 --_000_DM6PR13MB27624F07A612BDCF98A8C92F9A939DM6PR13MB2762namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song ; detnet@ietf.org; spring@ietf.org= ; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_DM6PR13MB27624F07A612BDCF98A8C92F9A939DM6PR13MB2762namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Yaakov,

Some feedback inline.

Best regards,

Haoyu

From: Yaakov Stein <yaakov_s@rad.com> <= br> Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu.song@futurewei.com>; detnet@ietf.org; sp= ring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Haoyu,

I’ll address your points:

> The use of clock time as deadline requires netw= ork synchronization

That is a basic assumption of TSN networks (IMO the = defining assumption).

However, the sync needn’t be highly accurate &= #8211; it depends on the tightness of the delay budgets.

>> Yes, I understand it. What I mean is that t= he method mentioned in the draft seems to be also applicable to other types= of networks. For example, we can envision some time-critical traffic in DC= N or WAN. If a protocol would be developed, can it also serve the other networks? If so, it would be better.

> accurate measurement of per-link propagation ti= me

Yes, I am assuming that once an for all someone does= a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measurement of the links, a= nd these are stored in some database.

Once again, the required accuracy depends on the del= ay budgets.

> which can somehow limit the application scope o= f this work

If the delay budget only above the physically minima= l delay by say less than 100 microseconds,

I agree that the previous two issues MUST be carried= out. But in such cases there is no alternative.

If the delay budget is much higher than that, then o= ne could use an RSVP-like mechanism,

sending a packet (or several packets) from source to= destination collecting a stack of timestamps,

and then using that stack for the following packets.=

> Mechanism should be provisioned to track where = the timing requirement is violated and by how much

I’ll leave the OAM for later. However there ar= e already many high accuracy performance measurement techniques and protoco= ls for this.

>> For this I mean something recorded in the s= ame packet with the deadlines. If it misses the deadline, the receiver may = need to know where it’s violated. Other independent methods are possi= ble, but it’s better to consider if it can be integrated in the current proposal.

> Recently programmable scheduler research has pr= oposed several primitives

Yes, I tried to stress that this ID is not limited t= o EDF (although sometimes that is a good strategy).

One can even reproduce Qbv behavior using a stack of= deadlines (although why would one wish to do so?).

> such as PIPO and PIEO

I’ve heard of PIFO (Push In First out) but not= PIPO. Is this a typo or something new?

I agree that there are mechanisms that are optimized= for hardware, but I have come up with a very nice hardware implementation = for PEDF

and prefer to find hardware implementations for opti= mal schedulers, rather than to determine schedulers based on optimal hardwa= re.

>> Sorry that’s a typo. I mean PIFO (alt= hough we do have a paper under review using the name PIPO). Yes I agree tho= se are just abstract primitives. The actual implementation, if customized t= o a particular algorithm, would be simpler.

Y(J)S

From: Haoyu Song <haoyu.song@futurewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Just got a chance to read your draft. I agree with t= he comments of the others that this is a very interesting work. I’ll = just add a few points.

The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget a device latency which require a rout= er/switch to obey. In case the overall budget is evenly divided by the hops= , a single parameter is enough. Of course, if one wants to customize the bu= dget on each hop (which might be necessary considering the different capability/capacity of each hop), a st= ack is still needed.
Mechanism should be provisioned to track where the ti= ming requirement is violated and by how much (e.g., using timestamp or flag= ). This would be very useful for troubleshooting.
Rec= ently programmable scheduler research has proposed several primitives such = as PIPO and PIEO and provided feasible hardware implementations. The scheme= proposed in this draft can easily fit into these primitives.

Best regards,

Haoyu

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_DM6PR13MB27624F07A612BDCF98A8C92F9A939DM6PR13MB2762namp_-- From nobody Sun Mar 7 21:37:13 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1862C3A2539; Sun, 7 Mar 2021 21:37:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.799 X-Spam-Level: X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AFyEzSd8zLNV; Sun, 7 Mar 2021 21:37:08 -0800 (PST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2066.outbound.protection.outlook.com [40.107.22.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3D3DE3A2537; Sun, 7 Mar 2021 21:37:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EFMTBF0KDm+oPUc/XtVlYzwC1zpQt1rC6jft2Bltp0OOdfNQs5idC7zqF79f4AL2U8x/yO19N8g8DmZad5vcuQGnF3aYRTcrigCZ7vQiZ28q8kP0Qidy+SM77E8In52r1SBEqkRZOUTvnKCuyiIf3sqm4bIeBd1BJcqNiLH1VNV7mcAJw9rjrOJC+ePEhIRaPsI9o6U/DWeQAK1wmUSdg4s8A0ANAuYB+p8D4cBiHgYV1co8tk1ykenuAL1NLuYze7k/4Ot8CLywPN8bHaDrmxVSLsYmYumVXIwBaxym4nCcf81y8vbjpjON0huf5tybFqTYMGjOk+vsV9ocdXcEhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fBxxHqXwNj1pOwBTvUYedEqlc69nMHQR2U0Jp14mlbQ=; b=hZSPGO8gRG/lWE8j91+W/KTcKVfUR38W3j11BbogqrTHDJGqkUqBBCuubZ3DyduCEdB3bjLJWe1Hf8fn8A27o2UWsR2qaUUF0kgvlXZy6JRUB8a76rs56rzrDCRt8Bsd9TjTvwgNDgwMPvmRt73LCS1aEpv9T2pKKWYxVRMqGuzogO7p+3V3iI07F/U216B1X0e0xqvdG8cVdb8iZnsZfMmrfVjRHuTQYfDTjcnBegUVqshvqCa1tZAosCejQb4xA7BGssfhLqhVzO4ARsTfSuCbTn8XqDsWm7LJO1vQ0UCyT36gxiEYXDwd2Q8myYleROBEwLK2ifA4Uk6DExE5UA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fBxxHqXwNj1pOwBTvUYedEqlc69nMHQR2U0Jp14mlbQ=; b=EDHonHUcT37q2YqtkhQo1kAcJT6y3eaBkKEmgwU2zkn429LROhIz1ElUwEv+peREWGF2oVwsY00w1TLkPo9TPLKK5YHyIR2deFUZCesoijxpay/589DJWBL6VZPll1w7irH+ZWHQbqSFDuABe2osLKIJ7+saqHhv9SnavvhCSnA= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM0PR03MB4995.eurprd03.prod.outlook.com (2603:10a6:208:105::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Mon, 8 Mar 2021 05:37:05 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 05:37:05 +0000 From: Yaakov Stein To: Haoyu Song , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpA Date: Mon, 8 Mar 2021 05:37:05 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: futurewei.com; dkim=none (message not signed) header.d=none;futurewei.com; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: f3fd642b-d3c0-4308-4ac6-08d8e1f4351a x-ms-traffictypediagnostic: AM0PR03MB4995: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: iTqjeux1ATMpAxQ2jXa8QlN+L5w4et6iqP9LQ95+7MQ3g/iX2kJfrVCC8o7ZP2i+NL8trWeh5175szfeIrDTZVCpYrrN4ecWF7SxbgV8WxNvf4U2hquOjf6plua7WVSXwGECN/S0F4J56209rl1JDSiwu8Ip6TwprT7i4FcljA1hD/PCjk10Dxwv2f7RpQK5AK53B0ih2nJLILyP5p4Xp23pVazZ/QexiPG0ojF3fI8y6EALBANgNafKDfKbY0TmNe20DUaSeD2VeX44eZlykeeBDFdDOosdGIiCVuerNGhU5nD0kXMOvFykFhGBpunsfMAMvRgq7wuCDj+Jb2C3hSQnNyoUrprLZli13X0vrUM2uMxSQ8yuhyLJoQKpyP/Gg8yHZNAbEYuB6+zqK1JqejUTHKDL/IUCwOKXHA2HjMd1wKnp8Nfxt+mYfeGc2b1yg1SQSooxDzZvQMIsYIw8v1reQV3PmnWNr98OdrpQiRg2LHseFqxvqiDap++b6kHnjR61BnjVwQkTw0WMdLI8W7spvvUoPtEY8NBMw7iRfq8rQoA+gLq/iXLbh7C/zr6H6uZ9NDVkV4uLllMObTV1g48avxVqjUPFePSBZn2hhzk= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(376002)(39850400004)(346002)(366004)(55016002)(83380400001)(53546011)(9686003)(478600001)(33656002)(86362001)(66574015)(8936002)(8676002)(7696005)(966005)(316002)(6506007)(5660300002)(110136005)(166002)(66946007)(64756008)(76116006)(186003)(2906002)(26005)(71200400001)(52536014)(66446008)(66556008)(66476007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?+crKNjHdnrBWvMwImN9buCt6Yhdezq2lNrlX6FhwZX5PQnlBS75wgQ4t348U?= =?us-ascii?Q?Y/pi8cp3XOvdG7KnKpCNszIGfmOZm2EzYiH3NZ17wLzZSgZ+XQV4dXffy/5y?= =?us-ascii?Q?WbDcgOvhGcRJeosIvEQ/VjlqmGWTpVgmYlHwax6oG8zMAkT0W5FgyIAcfj4D?= =?us-ascii?Q?9dt6HWpeJnIwURNv2Abr2HMqrF6Y75py5p2G+n+urWQ+mojIdJ1v0tfhnd1t?= =?us-ascii?Q?VRPDARhtMUwWjBQ9CnXdyu+Rmg900cNdzjnd8o4uUgK8KccMRccStQV9qRmo?= =?us-ascii?Q?TpjWeVK+Yx8xzou7XvOoIzV8IBE8ksZgXiUVt6ZwEWvAMwGRy2+CjA44ADAr?= =?us-ascii?Q?rIBprIkIzeO6F8tOhp209Uc2LaEi6NOIvY70SHgpy8mWj0Fbvzl/IJSUg6TZ?= =?us-ascii?Q?pzLdlSKHrfjjatuE/+zCzviQ3e9F0XUiSAkoi54EnlpTAuqBw1/mUbly9F/u?= =?us-ascii?Q?t/PfIBv22jcqmlByVoPFV+/d+/QCwhU4rkUM8g01sATGzeNZcBisy2dc48pH?= =?us-ascii?Q?O4W9MADZsfM7SBLpf06LM87BvJzRCPLhhAojN+DkrtxX69C7lSv7MXvwDfmP?= =?us-ascii?Q?8638z6mfexTWHIhbijPf1l2iSLQDUV9Qu1g1BY+8bQLAIlGwILVItExLc9gs?= =?us-ascii?Q?yEQazIifGhqLcloO0j6T6Rp5NjX/0RQVviz8OXshFlG9IHBdfAjkP3A9fh8d?= =?us-ascii?Q?9kWtN0kXxjFvaOqsl0egJnktvK6EkWyu9CFWD3vWfTcgrZMrFyJB1/1dSLPW?= =?us-ascii?Q?P1ch21JAolz1GjCOKTjYraenWOAXaFS4iRfO4VjVS/DdwNAPfrxklxsut1II?= =?us-ascii?Q?Tpvl5Fav2vBVSqqzAhDpwTBqb+P3QzFdtF7A4mGfAo7YblgAQvYtsHJ7nn/7?= =?us-ascii?Q?Mffoy4H8Ec9fpInwufLoLkvpfzXEHu04hA5+eEmjQekKzicH89feTb9sgE5D?= =?us-ascii?Q?EDxcimawuf/x583zWEBWT8W1WiCyHaOWLGtOx8/Y+5OLFuoW8TQqJPPEh28W?= =?us-ascii?Q?FOZieTDoYLshtfWakbZnyz1YQqH8nl1X7rPRd0PvrG9IBnR34xpYw/6rNdUj?= =?us-ascii?Q?jObwv/aul8PkRVwBffMzGhuQZSvHM0/KXV/Ye2ImzMk2v6QeXV7/bELTU69c?= =?us-ascii?Q?CYcgwqTuNPJglx1wnkApElkqIft739DTB7AJGxKqcrsFvLV9RdVudjvZsKHF?= =?us-ascii?Q?NqyudQJfQTy6eggrt550WfJsu9MngxeF3rulyWpm6P/R8nmTnlms+einc+nz?= =?us-ascii?Q?fSaTjkiXY8tMjpxF9U7XhEGh6P4fD6n+Q+rJR7u6lrEiXUNkNoqjHl4nQKvo?= =?us-ascii?Q?/+0aNt1VsF5KY49J9doJsfZr?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR03MB35223AD654033E1DE5963A21E5939AM0PR03MB3522eurp_" MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3fd642b-d3c0-4308-4ac6-08d8e1f4351a X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 05:37:05.1350 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GaNcy2D5137MwGNdD3BQ75p1MUImXPOqJWTDd0LS909gB5CVsTwCC5kYDq8Ow3RuPjkFvIBBs3O5kNRVPe5PSA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4995 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 05:37:11 -0000 --_000_AM0PR03MB35223AD654033E1DE5963A21E5939AM0PR03MB3522eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We'll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song Sent: 08/03/2021 04:41 To: Yaakov Stein ; detnet@ietf.org; spring@ietf.org; pce@= ietf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR03MB35223AD654033E1DE5963A21E5939AM0PR03MB3522eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Haoyu

I think we are in agreement.

I do not see the need for explicitly handling the ca= se of a packet missing a LOCAL deadline,

since the following switches will already handle thi= s case optimally (and may still meet the overall budget).

Counting packets that miss their delay budget is ind= eed important,

and a counter could be configured in the egress rout= er for this.

We’ll need to define this when we get to the p= rotocol specification.

It would be advantageous to put a threshold on the f= ailure rate

and feed this back to the path/stack optimizer.=

Y(J)S

From: Haoyu Song <haoyu.song@futurewei.com= >
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@rad.com>; detnet@ietf.org; spring@i= etf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Some feedback inline.

Best regards,

Haoyu

From: Yaakov Stein <yaakov_s@rad.com>
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Haoyu,

I’ll address your points:

> The use of clock time as deadline requires netw= ork synchronization

That is a basic assumption of TSN networks (IMO the = defining assumption).

However, the sync needn’t be highly accurate &= #8211; it depends on the tightness of the delay budgets.

> accurate measurement of per-link propagation ti= me

Yes, I am assuming that once an for all someone does= a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measurement of the links, a= nd these are stored in some database.

Once again, the required accuracy depends on the del= ay budgets.

> which can somehow limit the application scope o= f this work

If the delay budget only above the physically minima= l delay by say less than 100 microseconds,

I agree that the previous two issues MUST be carried= out. But in such cases there is no alternative.

If the delay budget is much higher than that, then o= ne could use an RSVP-like mechanism,

sending a packet (or several packets) from source to= destination collecting a stack of timestamps,

and then using that stack for the following packets.=

> Mechanism should be provisioned to track where = the timing requirement is violated and by how much

I’ll leave the OAM for later. However there ar= e already many high accuracy performance measurement techniques and protoco= ls for this.

> Recently programmable scheduler research has pr= oposed several primitives

Yes, I tried to stress that this ID is not limited t= o EDF (although sometimes that is a good strategy).

One can even reproduce Qbv behavior using a stack of= deadlines (although why would one wish to do so?).

> such as PIPO and PIEO

I’ve heard of PIFO (Push In First out) but not= PIPO. Is this a typo or something new?

I agree that there are mechanisms that are optimized= for hardware, but I have come up with a very nice hardware implementation = for PEDF

and prefer to find hardware implementations for opti= mal schedulers, rather than to determine schedulers based on optimal hardwa= re.

Y(J)S

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Just got a chance to read your draft. I agree with t= he comments of the others that this is a very interesting work. I’ll = just add a few points.

The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget a device latency which require a rout= er/switch to obey. In case the overall budget is evenly divided by the hops= , a single parameter is enough. Of course, if one wants to customize the bu= dget on each hop (which might be necessary considering the different capability/capacity of each hop), a st= ack is still needed.
Mechanism should be provisioned to track where the ti= ming requirement is violated and by how much (e.g., using timestamp or flag= ). This would be very useful for troubleshooting.
Rec= ently programmable scheduler research has proposed several primitives such = as PIPO and PIEO and provided feasible hardware implementations. The scheme= proposed in this draft can easily fit into these primitives.

Best regards,

Haoyu

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR03MB35223AD654033E1DE5963A21E5939AM0PR03MB3522eurp_-- From nobody Sun Mar 7 22:35:33 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF3A33A2626; Sun, 7 Mar 2021 22:35:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.348 X-Spam-Level: X-Spam-Status: No, score=-2.348 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tTak-FwmwBYx; Sun, 7 Mar 2021 22:35:28 -0800 (PST) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10080.outbound.protection.outlook.com [40.107.1.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 419753A2625; Sun, 7 Mar 2021 22:35:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cw4M3PY+msrPFnaHnmiSSXOWtUUB3MzT6xYTs6QPUFi1I+2MNUgKOPTvLp/88WSl28Qw00alXsjiCBgC0lm2RlKVaB6pGg1yyJd0vky52pXibF+WXLlpS5LDrY/RWqSyvqH4YoQdiyKKUEUIeloAz55u8rrVNK2DEmj3YeZqm4yPHkXaR8E09tB7Od+fouAbR2bgRGmP/1wYovmTm9x4lhRNX3LhGoJPXx0Qm5lu4SemQpuI7J8yT2/oqNXxNPqg6qW9l5v7xAFTMZz4UPCPuoVhDhNtEJamwuIo19BcRdM+DnWY0AH3D3L5IT74Q4NyUKNbfnACm9FKqRxxtf2KpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AMdpBOVf+Bl1DY+RJgvRePRQXKHI+H+O7fpH0hY1v3M=; b=B8jCdGKr0Cblxz8csUgBtgXmAtsLcS7iceBcuXBCE627uDNjut0dQ2bmWqI/isTBXQgkQvRvRoSJ2Nhi0UWAb1paZp3soZachMZi59LIzo/v1OgBOqkhg2kf7y7PfLgzbvsTP6DTwbO4Wtb0Q1IcesKzMLgE45S7x4eD/vf1VC2SD0eD5U2CIMxBICodl4yRfGJ6qDVSvmmAqiYBSOf+T1cWc6QfYBhy516hCdQ2hFjQVVXJeY5qA9BBgp+yko5jkh2I/vWsur8bt8yhdAAMDpcEQp19BrMh8/TwRYK0WWg0zJGP1Ei72Kue9cEMmfN3ogrRGzMgXNA5CS1u3XfPcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AMdpBOVf+Bl1DY+RJgvRePRQXKHI+H+O7fpH0hY1v3M=; b=o7j88tVzB5Tsp6S58oArPpngZt6eEL8/ap0tADAIKU3Vsm7O0w755JKdbXkKyDQFy38oDwJTF4xurdMGrXoAhOxrRZy2IIl1BYSTN4Io/mXtQEgPIHEPox264w9UcQDcZPVWcStOczL4kaTXpaPlXJB8DECJkE8ZgZ/U0IgrJIQ= Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com (2603:10a6:208:22::25) by AM4PR07MB3059.eurprd07.prod.outlook.com (2603:10a6:205:4::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.9; Mon, 8 Mar 2021 06:35:25 +0000 Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::8d2d:fd78:9fa:5f92]) by AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::8d2d:fd78:9fa:5f92%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 06:35:24 +0000 From: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= To: Yaakov Stein , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUACYqgyAAeeRYCA= Date: Mon, 8 Mar 2021 06:35:24 +0000 Message-ID: References: In-Reply-To: Accept-Language: hu-HU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: rad.com; dkim=none (message not signed) header.d=none;rad.com; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [185.29.82.144] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9e2ed9a4-3b34-49dc-8d89-08d8e1fc5ade x-ms-traffictypediagnostic: AM4PR07MB3059: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YDBmprAq9HAYDJ7nR04eJ7sJYCMp3DWebj9dwtaIux0JuAm0leem4El/D8UhoObdAH73eVPieYHpmRAmNSi5k2pgRWiCy1/TuKg5cIxJZmUy2TloLkEm8EFNcHeSqxsBMd+oLjuXn0NDe37NJkzMRjD2F6UpqSx0m7bCqlPFBxrVORluUUeRH8zj0L0KRmnwXUd7J2EhPhSqv9W0QTUrCdrCrx+IuwAKAxw+gqoFYwW+it6NcWnmGEUJuDJYrx+zaIqhnwFD7lpOnbBWYxos1dl2PqO8kE46cn3cRL/qgTD2ngg9PdmvN9mQ3b2/SxVeqzzLrvzj2AeYUXG6Nzsab/kaPNMWiNK5zuG7JqDiqxDP+pW8EQ8wnbr0VwU4cVw77XNGPbFx/6QWta9TUMWOHyNURoHLJAj5ZlkXfOG5ndVq2y30JQDFHQoWlB82n7FYViqc6FMj/kfhy6g4URgHi0BRtJ3MhTyAtoLysKgarHj3VNKQWHbLigxvBiMsek+SqJMneqKO7i3Xtyqi+/0HdesX9xCU8wz61nMSjwJfuwOKDPTd38zAioPg+n37klN8aoPIbRdwElCH3+calle4UzkNGCEzZW3kWSLLy3arzg4= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0702MB3603.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(396003)(39860400002)(366004)(376002)(33656002)(966005)(55016002)(8936002)(66556008)(86362001)(110136005)(83380400001)(71200400001)(186003)(9326002)(478600001)(316002)(66946007)(66574015)(8676002)(52536014)(64756008)(76116006)(5660300002)(166002)(7696005)(6506007)(53546011)(26005)(66476007)(9686003)(2906002)(66446008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?aQYgoS7TLA74Yo1A4vT31OuRYF7VsK6rbWv8cZ0Uefv7FQB+OYE71surhw?= =?iso-8859-1?Q?fDja9OrlermkmMRSx4WvPcnthGI/Q6OyGuc9NkzfvUhGmsSLtSmDgflbsC?= =?iso-8859-1?Q?SUODUxpgithVdh0+kunR3CGSk4yf8UsFuC/OpWiM/wZ+m3MDaR1jvKBRlI?= =?iso-8859-1?Q?bl4JJB61l9JcPXMP6wBdJYaN62g4MAowFe15nD6vjhumU3EPXIlkWoOHUJ?= =?iso-8859-1?Q?lClJxwRHueAq6j9FXATCm6qmOxHQXTNT4BsVmuONR6ylsCvGTENOkQz4OC?= =?iso-8859-1?Q?OstZB+xadJL6dn/kOV3VGATPRmXCkLqUtKWedqsdOO7yVF3VLcbksWpJsx?= =?iso-8859-1?Q?64OSV/nKaezgIEmgKrPJIi2UKldovZwF3CGp5iNJ4wpA6Ev9wF+OlUwlkm?= =?iso-8859-1?Q?hSd3KJSNVXugMCMjANRGLpnLNDT6FQEhL5jXHBtQHvpqLDInG5C48BoGCu?= =?iso-8859-1?Q?QkzgH8DmksGgi08dSrvm4KAVoGQLEt2aaWP+B/Z9cvqHEa9kei9CECFNPu?= =?iso-8859-1?Q?6uj68xy2RLOoMx31keF7lVJIVv5qm1/rqNQUdtYBRVQCqDWIAMZsI/HIOv?= =?iso-8859-1?Q?NzRc/QQhN128JHpjO0nobSXAVUTMOgJKwjgxtAS4wHpAuSfGuJ+hU/joYj?= =?iso-8859-1?Q?Hbq7qOgjyRDqdvgoAmCZ2khQKb50+TiC1uuM1tvKBTmtnBh7cjwkRPOIVX?= =?iso-8859-1?Q?PgNkIbYWZkh0sdUc8Q5KanCtZDAAl/Bc5weYquqW6naQtlB1THT2S4Cgpb?= =?iso-8859-1?Q?E5j9K73t1nXHzmKrThFScams3QJBmuh3QVXl6LUPD4yCQX3Hw7afANdBbJ?= =?iso-8859-1?Q?LoMSGfNcyBQaJ79JqvbTH880FM8ZBBk1Z5D56KQyg0LNH6wvTHHkbTjggS?= =?iso-8859-1?Q?PYYji07fw1IgpyrMcBmD/ru+QhI1t5BPt6ATchJKV76EUsmiV/W/YeJ9cA?= =?iso-8859-1?Q?fcx9LLIJ/c+37LTZEWiDR4dikZ/+X2P+S5ZfQBkJGLJuIQvKW6wuUySlNX?= =?iso-8859-1?Q?FP8SYttCUzQfFFwBkwE/NxYQjtPp2hJdGbqI5oLSgdPl5frZtOKuTDxLwz?= =?iso-8859-1?Q?h26aHMoFgWGh0VDBgKzvvO7CrCsfPZIDWSNunfvXy4eG48flpjklmJ3nbG?= =?iso-8859-1?Q?OQFqpBcy/EQ5HX714TFbufKnvzbmAsE2nzdHa/x1HaQ4hkYl2r0Ic+Vay3?= =?iso-8859-1?Q?TjI1hOAxdFvdRK1cxnPzry2EGP1GuS3QVB9ISH5ZqUMGSXXUu7FbnvnU1o?= =?iso-8859-1?Q?5HCAkdyBwIQwGyB20POQuriXmJ9r3AXi0oiRdH3iZPOGWFl85H8J9qHPsz?= =?iso-8859-1?Q?hrJJzoqUXE0M2ta+og9OYzohZInP66bRsUr/iEZi43g9o4q88SuCu2sjGC?= =?iso-8859-1?Q?7RfLt93MAa?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR0702MB3603772CE3B68FBB0F3F5D23AC939AM0PR0702MB3603_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR0702MB3603.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e2ed9a4-3b34-49dc-8d89-08d8e1fc5ade X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 06:35:24.4528 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: o8XMeRojbyAtDtFTX6OkAjt6dDg/T1vKb1iFnIH+Nu49sQKpp3axxntDw9WPoHvh9pIcho5o/j1hOJH/BqXqfSbEl2lRBNJlBMkeFxNxgJ8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR07MB3059 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 06:35:32 -0000 --_000_AM0PR0702MB3603772CE3B68FBB0F3F5D23AC939AM0PR0702MB3603_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Yaakov, Any view on these comments? Thanks Bala'zs From: detnet On Behalf Of Bal=E1zs Varga A Sent: Friday, February 26, 2021 3:03 PM To: Yaakov Stein ; detnet@ietf.org; spring@ietf.org; pce@= ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Hi Yaakov, many thanks for this well-written and information-rich draft. I have enjoyed to read it (and also the discussions on the lists). :--))) Conceptual (clarification) questions: 1, What is the node behavior if deadtime expires? Drop? That requires a qui= te good design/allocation of the latency budget. Otherwise it may result in= unnecessary loss of a packet being relative late at a hop, however that mi= ght be compensated by the remaining hops. (E.g., see your Figure 2, getting= 35 microsecond at R1 instead of the designed 25, but R2,R3,R4 adds only 10= -10-10 extra microseconds). You may consider to add a "late flag" to the pa= cket instead of a drop ... 2, Do I understand it correctly that the described solution is practically = a new per-hop-behavior? Does it have no tools to "influence" the arrival ti= me at the next hops? In end-to-end latency design the DetNet/TSN functions = are often used to "adapt" the arrival of packets belonging to a DetNet flow= /TSN stream at given network points. If "latency design" requires such a "t= ime shift" then the described method has to be combined with other DetNet/T= SN tools. 3, Design of a guaranteed upper bound usually requires deterministic arriva= l at each node along the path. In this solution the flow becomes less-an-le= ss deterministic as it reaches the destination. I mean e.g., as per Figure = 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-1= 20 usec and at R4 is 92-167 usec. So, the flow is getting less-and-less det= erministic at each hop. That can make deterministic design of other flows p= assing R4 more difficult or even impossible in some scenarios. Have I misse= d something here? 4, Wire-speed timestamping and calculation: I would be interested in your v= iew regarding how realistic are (1) the wire speed timestamping capability = and (2) adding multiple calculated deadline to packets. Both are required f= or this solution. DetNet flows can have high bandwidth, not like 1588 packe= ts. Specific comments (Section 2-3): 5, Time gated Queuing [802.1Qbv] allows multiple gates to be open simultane= ously. Furthermore preemption [802.1Qbu] can be combined with Qbv. So I can= not fully agree with your evaluation of time-aware scheduling. (Section 2, = "However, time-aware scheduling suffers from two major disadvantages.") Of = course with a bad design one may shoot in his/her own leg, but in well-desi= gned scenarios efficiency loss can be eliminated without expensive computat= ion. 6, Deadtime calculation is also complex if impact of other DetNet flows mus= t be considered. The pre-calculation of individual "local" deadlines may ne= ed to be re-calculated when flows added to the network and they are using s= ome common links. So I cannot agree with your statement. (Section 3, "... S= ince the ingress router inserts the deadline stack into the packet headers,= no other router needs to be aware of the requirements of the time sensitiv= e flows. Hence admitting a new flow only requires updating the information= base of the ingress router."). Adding a flow may result in updating many i= ngress routers' configuration. Some topics for further discussions/considerations: 7, Impact of some DetNet functions (e.g., PREOF [RFC8655]) needs further di= scussions. The usage of replication/elimination impacts the deadline calcul= ation. Disjoint paths used by member flows requires different label stack a= nd deadlines. Furthermore, these path specific labels+deadlines must be add= ed by the replication node (PRF). So at least, a combination of PRF with B-= SID and computing/adding deadlines (ingress SRTSN function) seems to be nec= essary in your solution. 8, The ever-green topic ( and not the green-wave :--))) ): label stack size= . You need multiple labels per hop and the label stack contains them for ea= ch hop. Could result in a quite big label stack to be pushed at the ingress= (nx10s of labels). I am happy to have further discussions on this interesting idea Thanks & Cheers Bala'zs From: detnet > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 2:14 PM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [Detnet] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR0702MB3603772CE3B68FBB0F3F5D23AC939AM0PR0702MB3603_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Yaakov,

Any view on these comments?

Thanks

Bala’zs

From: detnet <detnet-bounces@ietf.org> = On Behalf Of Bal=E1zs Varga A
Sent: Friday, February 26, 2021 3:03 PM
To: Yaakov Stein <yaakov_s@rad.com>; detnet@ietf.org; spring@i= etf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

many thanks for this well-written and information-ri= ch draft.

I have enjoyed to read it (and also the discussions = on the lists). :--)))

Conceptual (clarification) questions:<= /b>

1, What is the node behavior if deadtime expires? Dr= op? That requires a quite good design/allocation of the latency budget. Oth= erwise it may result in unnecessary loss of a packet being relative late at= a hop, however that might be compensated by the remaining hops. (E.g., see your Figure 2, getting 35 microsecond at= R1 instead of the designed 25, but R2,R3,R4 adds only 10-10-10 extra micro= seconds). You may consider to add a "late flag" to the packet ins= tead of a drop ...

2, Do I understand it correctly that the described s= olution is practically a new per-hop-behavior? Does it have no tools to &qu= ot;influence" the arrival time at the next hops? In end-to-end latency= design the DetNet/TSN functions are often used to "adapt" the arrival of packets belonging to a DetNet flow/TSN= stream at given network points. If "latency design" requires suc= h a "time shift" then the described method has to be combined wit= h other DetNet/TSN tools.

3, Design of a guaranteed upper bound usually requir= es deterministic arrival at each node along the path. In this solution the = flow becomes less-an-less deterministic as it reaches the destination. I me= an e.g., as per Figure 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-120 usec an= d at R4 is 92-167 usec. So, the flow is getting less-and-less deterministic= at each hop. That can make deterministic design of other flows passing R4 = more difficult or even impossible in some scenarios. Have I missed something here?

4, Wire-speed timestamping and calculation: I would = be interested in your view regarding how realistic are (1) the wire speed t= imestamping capability and (2) adding multiple calculated deadline to packe= ts. Both are required for this solution. DetNet flows can have high bandwidth, not like 1588 packets.

Specific comments (Section 2-3):
5, Time gated Queuing [802.1Qbv] allows multiple gat= es to be open simultaneously. Furthermore preemption [802.1Qbu] can be comb= ined with Qbv. So I cannot fully agree with your evaluation of time-aware s= cheduling. (Section 2, "However, time-aware scheduling suffers from two major disadvantages.") Of course with a b= ad design one may shoot in his/her own leg, but in well-designed scenarios = efficiency loss can be eliminated without expensive computation.=

6, Deadtime calculation is also complex if impact of= other DetNet flows must be considered. The pre-calculation of individual &= quot;local" deadlines may need to be re-calculated when flows added to= the network and they are using some common links. So I cannot agree with your statement. (Section 3, "... Since = the ingress router inserts the deadline stack into the packet headers, no o= ther router needs to be aware of the requirements of the time sensitive flo= ws. Hence admitting a new flow only requires updating the information base of the ingress router."). Addi= ng a flow may result in updating many ingress routers' configuration.<= /o:p>

Some topics for further discussions/consideration= s:

7, Impact of some DetNet functions (e.g., PREOF [RFC= 8655]) needs further discussions. The usage of replication/elimination impa= cts the deadline calculation. Disjoint paths used by member flows requires = different label stack and deadlines. Furthermore, these path specific labels+deadlines must be added by the rep= lication node (PRF). So at least, a combination of PRF with B-SID and compu= ting/adding deadlines (ingress SRTSN function) seems to be necessary in you= r solution.

8, The ever-green topic ( and not the green-wave :--= ))) ): label stack size. You need multiple labels per hop and the label sta= ck contains them for each hop. Could result in a quite big label stack to b= e pushed at the ingress (nx10s of labels).

I am happy to have further discussions on this inter= esting idea

Thanks & Cheers

Bala'zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 2:14 PM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [Detnet] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR0702MB3603772CE3B68FBB0F3F5D23AC939AM0PR0702MB3603_-- From nobody Sun Mar 7 23:26:58 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 868393A26E4; Sun, 7 Mar 2021 23:26:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.799 X-Spam-Level: X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UbJQl0lj1Crp; Sun, 7 Mar 2021 23:26:49 -0800 (PST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00045.outbound.protection.outlook.com [40.107.0.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 109FB3A26E3; Sun, 7 Mar 2021 23:26:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HfMasS+s+tc6GSrigapRXsUWQ/wA+Fudtgp23Z10yiH1G2BPb5Vw3urltAxvfYIBQFCF5BVD3uRcR87ahSYj98KlRST5PXk3ckoe5xJP4Ox2Fq7D4blSPZui/n3Rz9JSy9mjv1StJi7xtfkWtr1EAfu5L8dqSKsnBs+5rGQC1ZQduWQUh4q/PdIx80o/c1qDj2HURD5dteiP+4YVbDTGxQ3RPK3MaaX7zhl2gel7LG/ikgHQ0iTe41rZnXTznAbrWH9suAGYaqXEagpGD7sgmuCvKxLopIT04VfNrbDyp3Urs/ZpidQys9OSRSaWjELtnYmdXsyTECV+k4tgNolsqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O9Mi5RPLa5YbXUh/ITnQslydAgSd+6+UVsKP0xxDdtk=; b=Sl5jM7lDjfoiM7N5uo+e8vX2+surheDizQPA31e01mkiG7YNPs+ZQLXgV/embELHHRZqNHoulqhCLPqSCmFWQA13R3MAQrUDIK0XLYqlgyNvnnPfDAEeqmPy5J8fVBcyuFwL30iJmrZxK/NNLqaXARUXCWvJBqCj80YfQ1Gm07CloEhU/XGLnE/650eIF3P5T8detJG7GT0tfUwmM4OaDBIZLg0KrXiuJdHU5x5xjK9RJ74zE6JZnhzY7WN5Qyyf4qVlHIfTKtJELJ3RRfNP1xC0jLrLRiZOxtLx1oYW+PmVJ1f86/B2UdlmdOeNqDFJTfRP4mKH6NJlCw0tvn4xyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O9Mi5RPLa5YbXUh/ITnQslydAgSd+6+UVsKP0xxDdtk=; b=R97hBjlBg5w6ApIg5cL4+eSxYv5MGKueXcH6+A8PCBv8nZVs+KUL4o+apMirg9oPpIs/MI9r+Q/feKNGHHkWsFe5ib/4zR6SJEJijrP18jARPnx/ReTIWCMgRX+GyEgkeJ8lWwDwWBBd/La4h6d8dCPij9r1wIM6YDbOSM1Y8zk= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM9PR03MB6849.eurprd03.prod.outlook.com (2603:10a6:20b:285::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Mon, 8 Mar 2021 07:26:46 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 07:26:45 +0000 From: Yaakov Stein To: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUACYqgyAAeeRYCAAAGQfkA== Date: Mon, 8 Mar 2021 07:26:45 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ericsson.com; dkim=none (message not signed) header.d=none;ericsson.com; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 4952cca2-6310-448e-5525-08d8e2038787 x-ms-traffictypediagnostic: AM9PR03MB6849: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: g7ZkC3Y1GeEsGYQvCTn8Wx7k7iKlRaE+yEu37zXVkcAXinMYO+7cjEPMkLgO/+ozEKT/SXwE0F+dlq9vpn6wt1RZBZ58mU2ix4i3DaWr2anSpuJTk0NNuVXEk2XP3kBHmhzIBTupa3JqU/d886e2DnnwqjJqzU9e7+8E7FxxPsvw+K6XzX0I8T7TUPioYE+IDDRdQIY1qFuJ5zmnPcxN92unPzl2iwHhr08P5jcu+3molbBHOv/q/CnPSDX6U5EGQ1UM0eLgAr4yN5z1dlzkL184bf96lrCC1z9CzeB2tAlPpZZ+RJXkqxRDZn+qChQouTjKPjZjOfmfXdSz6sq9IEbZ0uPjk2TIPsb++06zygs61GXOhvnVYfwusTHOiFS4aoOFHOZZMARs9By8ZHvASbbrCNIeetGhJ35yffqk+fswxfF+q2TWhR4ftJcdMKMaR6fEQmlv0ItGs085Q6xX5RpSxh9cwziWt17TUrYgBEj2GjVbDMK/Mim8Jkwu408cnmrwGXe2+MzCkGHZTdJyrUxQ56NSySbsX34DHxx/cpM5N+jgeKxjMsdujMDs4UrdFZtdGT3XKWyLbbr1Ziy7WF5PyvnIGul0TaBVWBlhR2s= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(346002)(376002)(396003)(39850400004)(66574015)(71200400001)(26005)(186003)(2906002)(30864003)(5660300002)(33656002)(316002)(55016002)(8676002)(76116006)(9686003)(66946007)(83380400001)(52536014)(66556008)(64756008)(66446008)(110136005)(66476007)(166002)(8936002)(7696005)(86362001)(6506007)(966005)(478600001)(53546011); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?B4JhgGrSUfJLDYnj2b20QPaJaPnqv9O6nn+Y0I2u2wEvKGuR/01kRQsoP0?= =?iso-8859-1?Q?Fi4A88muPem1XvsN/z8m8KD92TuGb8+7IDH0BhFYnnGg2MnkbK+p3MdAjt?= =?iso-8859-1?Q?1wKk/hkHICz3Ey7MDgN7QEaEznYLo2WEe6RJXVp1fqqj9jFUSqyuPI6Cqx?= =?iso-8859-1?Q?kguw30XINNYiifi1Ff3XVbGk442EHL09MOpWrimwKcHVymjKR3pHuNMVCH?= =?iso-8859-1?Q?6xqp21wrnW3VvHhoJew5dfFuIANNFVW/86Cj7eVu17O0pYE4GkVLq6PQI2?= =?iso-8859-1?Q?SzShXE9L1Ai9g42wAn8pocqa5HTo567y42H1Nk4VBDixROsA/1u4QbOQav?= =?iso-8859-1?Q?dS5GLmOKF/sqDi8FlyJ5M0jFn+aT+xqzMZjVqeTYOHDFl3UYT+aDOsK2al?= =?iso-8859-1?Q?kIamO1kNL5m6v/V3/qakyFrjcJl6tg3DPJfTMfZRkY1P13rlHP2eSwL9Ox?= =?iso-8859-1?Q?c7QUcEUnhy5uPMDMsI4zJALf140CoUStCQhBXthIA/jQJELRXcemjZfICP?= =?iso-8859-1?Q?tZFivq8+rOJsDq4pYuQbgmd6Olgwy5jeP69VKMrYzaYbr64RTUZyKqsJef?= =?iso-8859-1?Q?qlbumBsrSlZIkkoJktRyScy+YNP51BB59dPt5pbUBtotaOnNiTsvhfNW+f?= =?iso-8859-1?Q?IDZj/flMFmGihINWUx9Ss4djK5b2akt93S+VBtxyav5kpTb58jlHGehHdL?= =?iso-8859-1?Q?fw/clp3IaVQxkIB6CWD0CSwFQh9Xegx6PKmX0tItgAMrCrMOnj2z/sPkzp?= =?iso-8859-1?Q?EChT9ndjNpcBRJao8s2WbE91Bzxg4+lYedLnAPXWItaLZOwmMkJIaaTBZp?= =?iso-8859-1?Q?FVhypq3pMQiBxNhCaC86d5A5eYCgM6236Vncuxp9jgMYslXzb620v/aqAD?= =?iso-8859-1?Q?shqdS7vCvoFmWgHW+1artRMmkgjMyjDlROtZ6o1gw1VP5uJc09DuZ1WSk/?= =?iso-8859-1?Q?8xFII5rLZZCzWhvkTJ4qhRkKXHb+JD7yEs60GA1Qir/gGy7858P5rmOTYA?= =?iso-8859-1?Q?zT/CbaLU/6oS7aeIRsUKqqJqpGsvWlmK6KGcat+kg9T0cmS+UWDbu6/N7V?= =?iso-8859-1?Q?YzrrTnBeg9mlzmcD5luvWl/qybVN35pMOiM3M6dVUVlO/OaalWRBfRe1z5?= =?iso-8859-1?Q?YLJOzPZhFga+WaCnTOmLFc3eO6LY3t9jeJpw44jqANQBU9+yiv6YJX29Yz?= =?iso-8859-1?Q?PdbH5w49SP40zm5qQ4X7LH0VlyUsZrvd64QO/EdGAIlUFSbYIrOvwklHyI?= =?iso-8859-1?Q?H1EtEja2iRzmJG8MeUijF2S5KFZnEApqRRpm9EaN/jV2Wtn1itKJd4zdLf?= =?iso-8859-1?Q?yGTQObHe0pKJEgqyL+n8hG9H4PD/PS7qzKqhup+6HNdQdh0Kd/R9QdD7Np?= =?iso-8859-1?Q?FMDwiTYISY?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR03MB3522FFDD0AE3080389660D86E5939AM0PR03MB3522eurp_" MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4952cca2-6310-448e-5525-08d8e2038787 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 07:26:45.8698 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: YjCpC2x17HrRyuMtKosVZFlU5LHMoIyXL5BrtWIV+F8DIl80O4Z/u8YQ2rrKu4RkG1NPwIBXjUr7wG6MP1rrFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6849 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 07:26:52 -0000 --_000_AM0PR03MB3522FFDD0AE3080389660D86E5939AM0PR03MB3522eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Bala'zs, Sorry that I never got back to you on this. It is much easier to shoot back replies to shorter emails :) > 1, What is the node behavior if deadtime expires? Nothing special is done if a local deadline is exceeded. There is still hop= e that the time can be made up. If too many packets fail to conform to their overall budget, then maybe something has changed and reoptimization (or realizing that the = application budget is not feasible) is required. > 2, Do I understand it correctly that the described solution is practicall= y a new per-hop-behavior? Well, new to here but hardly "new" (similar things were done in ATM...) The new per-router behavior requires a new data structure in the router ins= tead of a FIFO queue, but there is no absolute requirement for other tools. As I just hinted and have mentioned in a previous email, later on we may ne= ed some OAM to see if configuration needs to be updated. I have already experimented with periodic RSVP-like synthetic packets that = collect timestamps along the path. > 3, Design of a guaranteed upper bound usually requires deterministic arri= val at each node along the path. Here is where we differ (or at least put different emphasis on the word "us= ually". Determinism is not the main goal of this draft. Meeting challenging delay b= udgets is. Sometimes absolute determinacy is required (e.g., in some forms of "telepro= tection" in some electricity distribution networks) in which case either TDM/Qbv/calendaring can be used, or this method couple= d with a differential delay compensation buffer. I agree that PDV increases from hop to hop to an extent, but when the delay= budget is really challenging you simply can't allow that many hops. In mixed 5G xHaul cases in which I have been involved there are no more tha= n 4 hops (usually fewer). If you wish something more deterministic then look at the Andrews-Zhang str= ategy. They gate at the ingress router and EDF at all the following ones. They show that (statistically) they ride "green waves" and thus the PDV doe= sn't increase significantly (see their graphs). This comes at the expense of a long cycle time for the = initial gate, which means the budget can't be too tight. > 4, Wire-speed timestamping and calculation I come from the PTP world where hardware timestamping with submicro resolut= ion is common. Since the timestamp is on the rising edge of the first bit, i.e., before th= e classifier, it is carried out and placed in the packet metadata for every packet whethe= r or not it is PTP. I always shed tears over throwing out this information. The RSVP-like method just mentioned simply copies it into the packet. The SRTSN packets use this information in the scheduler. I have consulted my ASIC people about the use of priority heaps and insert-= queues and they ensure me that they can put something like that into our 100G chip= . > 5, Time gated Queuing [802.1Qbv] I didn't say that in principle you can't build a Qbv system that does great= things. It is flexible enough to do anything you want. The problem is the scalability. I have played around with optimizing it and haven't found a good "on line" = algorithm, that is, a way of adding a new flow to an existing optimized design. And even in the "off line" case (clean slate with all flows known) slight c= hanges and even bad clocks in the user equipment ruins things. But the main idea of this draft is that even given a good on-line algorithm= for Qbv you would need to distribute the changes in the all the schedules to all th= e routers. What happens in the meantime? What happens when one router is already updat= ed but another isn't? With the stack method only a single ingress router needs to be updated and = consistency is ensured! > 6, Deadtime calculation is also complex if impact of other DetNet flows m= ust be considered. Of course! The offset vector needs to be optimized. I have several such methods and am running simulations of various strategie= s. As stated in the ID the specific calculation was for illustration purposes = - the stack mechanism is much more generic. > 7, Impact of some DetNet functions I have not yet gotten to the point where I have considered this. > 8, The ever-green topic ( and not the green-wave :--))) ): label stack s= ize. Yes, I didn't want to bring this up yet, but have been forced to. I have to admit that I am not a fan of SRv6 for just this reason. But, the SRTSN stack can be really small! SR stack entries need only be ceil ( log2 (number of routers) ) in size (re= quires a little local information base in the router) assuming once the stack is popped you use the original DA. If you are not w= illing to use this additional information you still only need sufficient suffixes, which is still small. The deadline entries can be miniscule since the wraparound constraint is de= rived from the maximum flight time of a packet in the network. In my designs I have gotten away with 16-32 bits per entry, which means 4-8= hop stacks only require the room of a single IPv6 address. Hope this answers your questions and kicks off a fruitful discussion. Y(J)S From: Bal=E1zs Varga A Sent: 08/03/2021 08:35 To: Yaakov Stein ; detnet@ietf.org; spring@ietf.org; pce@= ietf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Any view on these comments? Thanks Bala'zs From: detnet > On B= ehalf Of Bal=E1zs Varga A Sent: Friday, February 26, 2021 3:03 PM To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Hi Yaakov, many thanks for this well-written and information-rich draft. I have enjoyed to read it (and also the discussions on the lists). :--))) Conceptual (clarification) questions: 1, What is the node behavior if deadtime expires? Drop? That requires a qui= te good design/allocation of the latency budget. Otherwise it may result in= unnecessary loss of a packet being relative late at a hop, however that mi= ght be compensated by the remaining hops. (E.g., see your Figure 2, getting= 35 microsecond at R1 instead of the designed 25, but R2,R3,R4 adds only 10= -10-10 extra microseconds). You may consider to add a "late flag" to the pa= cket instead of a drop ... 2, Do I understand it correctly that the described solution is practically = a new per-hop-behavior? Does it have no tools to "influence" the arrival ti= me at the next hops? In end-to-end latency design the DetNet/TSN functions = are often used to "adapt" the arrival of packets belonging to a DetNet flow= /TSN stream at given network points. If "latency design" requires such a "t= ime shift" then the described method has to be combined with other DetNet/T= SN tools. 3, Design of a guaranteed upper bound usually requires deterministic arriva= l at each node along the path. In this solution the flow becomes less-an-le= ss deterministic as it reaches the destination. I mean e.g., as per Figure = 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-1= 20 usec and at R4 is 92-167 usec. So, the flow is getting less-and-less det= erministic at each hop. That can make deterministic design of other flows p= assing R4 more difficult or even impossible in some scenarios. Have I misse= d something here? 4, Wire-speed timestamping and calculation: I would be interested in your v= iew regarding how realistic are (1) the wire speed timestamping capability = and (2) adding multiple calculated deadline to packets. Both are required f= or this solution. DetNet flows can have high bandwidth, not like 1588 packe= ts. Specific comments (Section 2-3): 5, Time gated Queuing [802.1Qbv] allows multiple gates to be open simultane= ously. Furthermore preemption [802.1Qbu] can be combined with Qbv. So I can= not fully agree with your evaluation of time-aware scheduling. (Section 2, = "However, time-aware scheduling suffers from two major disadvantages.") Of = course with a bad design one may shoot in his/her own leg, but in well-desi= gned scenarios efficiency loss can be eliminated without expensive computat= ion. 6, Deadtime calculation is also complex if impact of other DetNet flows mus= t be considered. The pre-calculation of individual "local" deadlines may ne= ed to be re-calculated when flows added to the network and they are using s= ome common links. So I cannot agree with your statement. (Section 3, "... S= ince the ingress router inserts the deadline stack into the packet headers,= no other router needs to be aware of the requirements of the time sensitiv= e flows. Hence admitting a new flow only requires updating the information= base of the ingress router."). Adding a flow may result in updating many i= ngress routers' configuration. Some topics for further discussions/considerations: 7, Impact of some DetNet functions (e.g., PREOF [RFC8655]) needs further di= scussions. The usage of replication/elimination impacts the deadline calcul= ation. Disjoint paths used by member flows requires different label stack a= nd deadlines. Furthermore, these path specific labels+deadlines must be add= ed by the replication node (PRF). So at least, a combination of PRF with B-= SID and computing/adding deadlines (ingress SRTSN function) seems to be nec= essary in your solution. 8, The ever-green topic ( and not the green-wave :--))) ): label stack size= . You need multiple labels per hop and the label stack contains them for ea= ch hop. Could result in a quite big label stack to be pushed at the ingress= (nx10s of labels). I am happy to have further discussions on this interesting idea Thanks & Cheers Bala'zs From: detnet > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 2:14 PM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [Detnet] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR03MB3522FFDD0AE3080389660D86E5939AM0PR03MB3522eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Bala'zs,

Sorry that I never got back to you on this.

It is much easier to shoot back replies to shorter e= mails :)

> 1, What is the node behavior if deadtime expire= s?

Nothing special is done if a local deadline is excee= ded. There is still hope that the time can be made up.

If too many packets fail to conform to their overall= budget,

then maybe something has changed and reoptimization = (or realizing that the application budget is not feasible) is required.

> 2, Do I understand it correctly that the descri= bed solution is practically a new per-hop-behavior?

Well, new to here but hardly “new” (simi= lar things were done in ATM…)

The new per-router behavior requires a new data stru= cture in the router instead of a FIFO queue,

but there is no absolute requirement for other tools= .

As I just hinted and have mentioned in a previous em= ail, later on we may need some OAM to see if configuration needs to be upda= ted.

I have already experimented with periodic RSVP-like = synthetic packets that collect timestamps along the path.

> 3, Design of a guaranteed upper bound usually r= equires deterministic arrival at each node along the path.

Here is where we differ (or at least put different e= mphasis on the word “usually”.

Determinism is not the main goal of this draft. Meet= ing challenging delay budgets is.

Sometimes absolute determinacy is required (e.g., in= some forms of “teleprotection” in some electricity distributio= n networks)

in which case either TDM/Qbv/calendaring can be used= , or this method coupled with a differential delay compensation buffer.

I agree that PDV increases from hop to hop to an ext= ent, but when the delay budget is really challenging you simply can’t= allow that many hops.

In mixed 5G xHaul cases in which I have been involve= d there are no more than 4 hops (usually fewer).

If you wish something more deterministic then look a= t the Andrews-Zhang strategy.

They gate at the ingress router and EDF at all the f= ollowing ones.

They show that (statistically) they ride “gree= n waves” and thus the PDV doesn’t increase significantly

(see their graphs). This comes at the expense of a l= ong cycle time for the initial gate, which means the budget can’t be = too tight.

> 4, Wire-speed timestamping and calculation=

I come from the PTP world where hardware timestampin= g with submicro resolution is common.

Since the timestamp is on the rising edge of the fir= st bit, i.e., before the classifier,

it is carried out and placed in the packet metadata = for every packet whether or not it is PTP.

I always shed tears over throwing out this informati= on.

The RSVP-like method just mentioned simply copies it= into the packet.

The SRTSN packets use this information in the schedu= ler.

I have consulted my ASIC people about the use of pri= ority heaps and insert-queues

and they ensure me that they can put something like = that into our 100G chip.

> 5, Time gated Queuing [802.1Qbv]

I didn’t say that in principle you can’t= build a Qbv system that does great things.

It is flexible enough to do anything you want.<= /o:p>

The problem is the scalability.

I have played around with optimizing it and haven= 217;t found a good “on line” algorithm,

that is, a way of adding a new flow to an existing o= ptimized design.

And even in the “off line” case (clean s= late with all flows known) slight changes and even bad clocks in the user e= quipment ruins things.

But the main idea of this draft is that even given a= good on-line algorithm for Qbv

you would need to distribute the changes in the all = the schedules to all the routers.

What happens in the meantime? What happens when one = router is already updated but another isn’t?

With the stack method only a single ingress router n= eeds to be updated and consistency is ensured!

> 6, Deadtime calculation is also complex if impa= ct of other DetNet flows must be considered.

Of course! The offset vector needs to be optimized.<= o:p>

I have several such methods and am running simulatio= ns of various strategies.

As stated in the ID the specific calculation was for= illustration purposes –

the stack mechanism is much more generic.=

> 7, Impact of some DetNet functions
I have not yet gotten to the point where I have cons= idered this.

> 8, The ever-green topic ( and not the gre= en-wave :--))) ): label stack size.

Yes, I didn’t want to bring this up yet, but h= ave been forced to.

I have to admit that I am not a fan of SRv6 for just= this reason.

But, the SRTSN stack can be really small!=

SR stack entries need only be ceil ( log2 (number of= routers) ) in size (requires a little local information base in the router= )

assuming once the stack is popped you use the origin= al DA. If you are not willing to use this additional information

you still only need sufficient suffixes, which is st= ill small.

The deadline entries can be miniscule since the wrap= around constraint is derived from the maximum flight time of a packet in th= e network.

In my designs I have gotten away with 16-32 bits per= entry, which means 4-8 hop stacks only require the room of a single IPv6 a= ddress.

Hope this answers your questions and kicks off a fru= itful discussion.

Y(J)S

From: Bal=E1zs Varga A <balazs.a.varga@eri= csson.com>
Sent: 08/03/2021 08:35
To: Yaakov Stein <yaakov_s@rad.com>; detnet@ietf.org; spring@i= etf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Any view on these comments?

Thanks

Bala’zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Bal=E1zs Varga A
Sent: Friday, February 26, 2021 3:03 PM
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

many thanks for this well-written and information-ri= ch draft.

I have enjoyed to read it (and also the discussions = on the lists). :--)))

Conceptual (clarification) questions:

1, What is the node behavior if deadtime expires? Dr= op? That requires a quite good design/allocation of the latency budget. Oth= erwise it may result in unnecessary loss of a packet being relative late at= a hop, however that might be compensated by the remaining hops. (E.g., see your Figure 2, getting 35 microsecond at= R1 instead of the designed 25, but R2,R3,R4 adds only 10-10-10 extra micro= seconds). You may consider to add a "late flag" to the packet ins= tead of a drop ...

2, Do I understand it correctly that the described s= olution is practically a new per-hop-behavior? Does it have no tools to &qu= ot;influence" the arrival time at the next hops? In end-to-end latency= design the DetNet/TSN functions are often used to "adapt" the arrival of packets belonging to a DetNet flow/TSN= stream at given network points. If "latency design" requires suc= h a "time shift" then the described method has to be combined wit= h other DetNet/TSN tools.

3, Design of a guaranteed upper bound usually requir= es deterministic arrival at each node along the path. In this solution the = flow becomes less-an-less deterministic as it reaches the destination. I me= an e.g., as per Figure 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-120 usec an= d at R4 is 92-167 usec. So, the flow is getting less-and-less deterministic= at each hop. That can make deterministic design of other flows passing R4 = more difficult or even impossible in some scenarios. Have I missed something here?

4, Wire-speed timestamping and calculation: I would = be interested in your view regarding how realistic are (1) the wire speed t= imestamping capability and (2) adding multiple calculated deadline to packe= ts. Both are required for this solution. DetNet flows can have high bandwidth, not like 1588 packets.

Specific comments (Section 2-3):
5, Time gated Queuing [802.1Qbv] allows multiple gat= es to be open simultaneously. Furthermore preemption [802.1Qbu] can be comb= ined with Qbv. So I cannot fully agree with your evaluation of time-aware s= cheduling. (Section 2, "However, time-aware scheduling suffers from two major disadvantages.") Of course with a b= ad design one may shoot in his/her own leg, but in well-designed scenarios = efficiency loss can be eliminated without expensive computation.=

6, Deadtime calculation is also complex if impact of= other DetNet flows must be considered. The pre-calculation of individual &= quot;local" deadlines may need to be re-calculated when flows added to= the network and they are using some common links. So I cannot agree with your statement. (Section 3, "... Since = the ingress router inserts the deadline stack into the packet headers, no o= ther router needs to be aware of the requirements of the time sensitive flo= ws. Hence admitting a new flow only requires updating the information base of the ingress router."). Addi= ng a flow may result in updating many ingress routers' configuration.<= /o:p>

Some topics for further discussions/consideration= s:

7, Impact of some DetNet functions (e.g., PREOF [RFC= 8655]) needs further discussions. The usage of replication/elimination impa= cts the deadline calculation. Disjoint paths used by member flows requires = different label stack and deadlines. Furthermore, these path specific labels+deadlines must be added by the rep= lication node (PRF). So at least, a combination of PRF with B-SID and compu= ting/adding deadlines (ingress SRTSN function) seems to be necessary in you= r solution.

8, The ever-green topic ( and not the green-wave :--= ))) ): label stack size. You need multiple labels per hop and the label sta= ck contains them for each hop. Could result in a quite big label stack to b= e pushed at the ingress (nx10s of labels).

I am happy to have further discussions on this inter= esting idea

Thanks & Cheers

Bala'zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 2:14 PM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [Detnet] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR03MB3522FFDD0AE3080389660D86E5939AM0PR03MB3522eurp_-- From nobody Mon Mar 8 01:38:41 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F2903A28C4; Mon, 8 Mar 2021 01:38:35 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2XrbDhmaKefQ; Mon, 8 Mar 2021 01:38:33 -0800 (PST) Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EF47A3A28C3; Mon, 8 Mar 2021 01:38:32 -0800 (PST) Received: from fraeml712-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DvCmh3wYqz67wJP; Mon, 8 Mar 2021 17:32:36 +0800 (CST) Received: from nkgeml701-chm.china.huawei.com (10.98.57.156) by fraeml712-chm.china.huawei.com (10.206.15.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Mon, 8 Mar 2021 10:38:29 +0100 Received: from nkgeml701-chm.china.huawei.com (10.98.57.156) by nkgeml701-chm.china.huawei.com (10.98.57.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Mon, 8 Mar 2021 17:38:27 +0800 Received: from nkgeml701-chm.china.huawei.com ([10.98.57.156]) by nkgeml701-chm.china.huawei.com ([10.98.57.156]) with mapi id 15.01.2106.013; Mon, 8 Mar 2021 17:38:27 +0800 From: "Yangfan (IP Standard)" To: Greg Mirsky , "draft-geng-6man-redundancy-protection-srh@ietf.org" CC: 6man WG , DetNet WG , Greg Mirsky Thread-Topic: Flow Identification in IPv6 Thread-Index: AQHXEsYqJXKpi/2re0+2TVz1hyjrnap5pPVA Date: Mon, 8 Mar 2021 09:38:27 +0000 Message-ID: <3fdc1006788e47e59cfb8dcc03e9bce6@huawei.com> References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.108.243.115] Content-Type: multipart/alternative; boundary="_000_3fdc1006788e47e59cfb8dcc03e9bce6huaweicom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: Subject: [Detnet] =?utf-8?b?562U5aSNOiBGbG93IElkZW50aWZpY2F0aW9uIGluIElQ?= =?utf-8?q?v6?= X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 09:38:35 -0000 --_000_3fdc1006788e47e59cfb8dcc03e9bce6huaweicom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGkgR3JlZywNCg0KTGl0ZXJhbGx5IHNwZWFraW5nLCBJUHY2ICBGbG93IExhYmVsIGNvdWxkIGJl IHVzZWQgdG8gaWRlbnRpZnkgYSBzcGVjaWZpYyBmbG93IG5lZWRpbmcgcmVkdW5kYW5jeSBwcm90 ZWN0aW9uIGluIFNSdjYgZGF0YSBwbGFuZS4gQnV0IEkgbWF5IGhhdmUgY29uY2VybnMgdGhhdCBm bG93IGxhYmVsIGNhbm5vdCBiZSBwcm90ZWN0ZWQgdG8gYmUgdW5tb2RpZmllZCBlbiByb3V0ZS4g QSBtb2RpZmllZCBmbG93IGxhYmVsIGNhbiBiZSBhIGRpc2FzdGVyIGZvciB0aGUgdHJhZmZpY3Mg IHdoaWNoIGFyZSBzZWVraW5nIGZvciBkZXRlcm1pbmlzdGljIGZvcndhcmRpbmcsIG5vdCBvbmx5 IHRoaXMgZmxvdywgYWxzbyBhZmZlY3Rpbmcgb3RoZXIgZmxvd3MgdXNpbmcgcmVkdW5kYW5jeSBw cm90ZWN0aW9uLiBBbmQgd2l0aCBzZXZlcmFsIHNlY3VyaXR5IGlzc3VlcyBtZW50aW9uZWQgaW4g UkZDNjQzNywgSSBkb3VidCBpZiBpdCBpcyBhIGdvb2QgaWRlYSB0byB1c2VyIElQdjYgRmxvdyBM YWJlbC4NCkp1c3QgbXkgMmNlbnRzIG9waW5pb24sIGhvdyBkbyB5b3UgYW5kIG90aGVyIHBlb3Bs ZSBzZWUgaXQ/DQoNClJlZ2FyZHMsDQpGYW4NCg0KDQoNCg0K5Y+R5Lu25Lq6OiBHcmVnIE1pcnNr eSBbbWFpbHRvOmdyZWdpbWlyc2t5QGdtYWlsLmNvbV0NCuWPkemAgeaXtumXtDogMjAyMeW5tDPm nIg35pelIDQ6MjANCuaUtuS7tuS6ujogZHJhZnQtZ2VuZy02bWFuLXJlZHVuZGFuY3ktcHJvdGVj dGlvbi1zcmhAaWV0Zi5vcmcNCuaKhOmAgTogNm1hbiBXRyA8aXB2NkBpZXRmLm9yZz47IERldE5l dCBXRyA8ZGV0bmV0QGlldGYub3JnPjsgR3JlZyBNaXJza3kgPGdyZWdvcnkubWlyc2t5QHp0ZXR4 LmNvbT4NCuS4u+mimDogRmxvdyBJZGVudGlmaWNhdGlvbiBpbiBJUHY2DQoNCkRlYXIgQXV0aG9y cywNCnRoYW5rIHlvdSBmb3IgYnJpbmdpbmcgeW91ciBwcm9wb3NhbCB0byB0aGUgZGlzY3Vzc2lv bi4gSSBhZ3JlZSB3aXRoIHlvdXIgdmlldyB0aGF0IHRoZSBleHBsaWNpdCByb3V0aW5nIGVuYWJs ZWQgYnkgU1J2NiBjcmVhdGVzIGFuIGVudmlyb25tZW50IHdoZXJlIFBSRU9GIGNhbiBiZSB1c2Vk LiBBbmQsIGFzIHdlIGtub3csIFRoZSBQUkVPRiBtYXkgYmUgdXNlZCBpbiBhIERldE5ldCBkb21h aW4gdG8gbG93ZXIgcGFja2V0IGxvc3MgcmF0aW8gYW5kIHByb3ZpZGUgYm91bmRlZCBsYXRlbmN5 Lg0KQWZ0ZXIgcmVhZGluZyB0aGUgZHJhZnQsIEkndmUgZ290IGEgcXVlc3Rpb24gZm9yIHlvdS4g V2hhdCBkbyB5b3Ugc2VlIGFzIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIElQdjYgRmxvdyBM YWJlbCBwZXIgUkZDIDY0MzcgYW5kIHRoZSBGbG93IElkZW50aWZpY2F0aW9uIGZpZWxkIGluIHRo ZSBUTFYgcHJvcG9zZWQgaW4gdGhlIGRyYWZ0PyBDb3VsZCB0aGUgSVB2NiBGbG93IExhYmVsIGJl IHVzZWQgdG8gaWRlbnRpZnkgdGhlIGZsb3cgZm9yIHRoZSBQUkVPRj8NCg0KUmVnYXJkcywNCkdy ZWcNCg== --_000_3fdc1006788e47e59cfb8dcc03e9bce6huaweicom_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 5a6L5L2TOw0KCXBhbm9zZS0xOjIgMSA2IDAgMyAxIDEgMSAxIDE7fQ0KQGZvbnQtZmFjZQ0KCXtm b250LWZhbWlseToiQ2FtYnJpYSBNYXRoIjsNCglwYW5vc2UtMToyIDQgNSAzIDUgNCA2IDMgMiA0 O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToyIDE1IDUg MiAyIDIgNCAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OiJcQOWui+S9kyI7DQoJ cGFub3NlLTE6MiAxIDYgMCAzIDEgMSAxIDEgMTt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5 OuW+rui9r+mbhem7kTsNCglwYW5vc2UtMToyIDExIDUgMyAyIDIgNCAyIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OiJcQOW+rui9r+mbhem7kSI7DQoJcGFub3NlLTE6MiAxMSA1IDMg MiAyIDQgMiAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBsaS5N c29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4tYm90dG9tOi4w MDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseTrlrovkvZM7fQ0KYTpsaW5r LCBzcGFuLk1zb0h5cGVybGluaw0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJY29sb3I6IzA1 NjNDMTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30NCmE6dmlzaXRlZCwgc3Bhbi5Nc29I eXBlcmxpbmtGb2xsb3dlZA0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJY29sb3I6Izk1NEY3 MjsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30NCnNwYW4uRW1haWxTdHlsZTE3DQoJe21z by1zdHlsZS10eXBlOnBlcnNvbmFsOw0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlm Ow0KCWNvbG9yOiMxRjQ5N0Q7fQ0Kc3Bhbi5FbWFpbFN0eWxlMTgNCgl7bXNvLXN0eWxlLXR5cGU6 cGVyc29uYWwtY29tcG9zZTsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fucy1zZXJpZjsNCglj b2xvcjp3aW5kb3d0ZXh0O30NCi5Nc29DaHBEZWZhdWx0DQoJe21zby1zdHlsZS10eXBlOmV4cG9y dC1vbmx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmO30NCkBwYWdlIFdvcmRT ZWN0aW9uMQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDkwLjBwdCA3 Mi4wcHQgOTAuMHB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0K LS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVmYXVsdHMgdjpl eHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0 ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxvOmlkbWFwIHY6 ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0t Pg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iWkgtQ04iIGxpbms9IiMwNTYzQzEiIHZsaW5rPSIjOTU0 RjcyIj4NCjxkaXYgY2xhc3M9IldvcmRTZWN0aW9uMSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48 c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7Zm9udC1mYW1pbHk6JnF1 b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPkhpIEdyZWcsPG86cD48 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMi IHN0eWxlPSJmb250LXNpemU6MTAuNXB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss c2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZTox MC41cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMx RjQ5N0QiPkxpdGVyYWxseSBzcGVha2luZywgSVB2NiZuYnNwOyBGbG93IExhYmVsIGNvdWxkIGJl IHVzZWQgdG8gaWRlbnRpZnkgYSBzcGVjaWZpYyBmbG93IG5lZWRpbmcgcmVkdW5kYW5jeSBwcm90 ZWN0aW9uIGluIFNSdjYgZGF0YSBwbGFuZS4gQnV0IEkgbWF5IGhhdmUgY29uY2VybnMNCiB0aGF0 IGZsb3cgbGFiZWwgY2Fubm90IGJlIHByb3RlY3RlZCB0byBiZSB1bm1vZGlmaWVkIGVuIHJvdXRl LiBBIG1vZGlmaWVkIGZsb3cgbGFiZWwgY2FuIGJlIGEgZGlzYXN0ZXIgZm9yIHRoZSB0cmFmZmlj cyAmbmJzcDt3aGljaCBhcmUgc2Vla2luZyBmb3IgZGV0ZXJtaW5pc3RpYyBmb3J3YXJkaW5nLCBu b3Qgb25seSB0aGlzIGZsb3csIGFsc28gYWZmZWN0aW5nIG90aGVyIGZsb3dzIHVzaW5nIHJlZHVu ZGFuY3kgcHJvdGVjdGlvbi4gQW5kIHdpdGggc2V2ZXJhbA0KIHNlY3VyaXR5IGlzc3VlcyBtZW50 aW9uZWQgaW4gUkZDNjQzNywgSSBkb3VidCBpZiBpdCBpcyBhIGdvb2QgaWRlYSB0byB1c2VyIElQ djYgRmxvdyBMYWJlbC4NCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEwLjVwdDtmb250LWZhbWls eTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+SnVzdCBteSAy Y2VudHMgb3BpbmlvbiwgaG93IGRvIHlvdSBhbmQgb3RoZXIgcGVvcGxlIHNlZSBpdD88bzpwPjwv bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIg c3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90Oyxz YW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEw LjVwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFG NDk3RCI+UmVnYXJkcyw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs Ij48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7Zm9udC1mYW1pbHk6 JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPkZhbg0KPG86cD48 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMi IHN0eWxlPSJmb250LXNpemU6MTAuNXB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss c2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZTox MC41cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMx RjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEwLjVwdDtmb250LWZhbWlseTom cXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMi IHN0eWxlPSJmb250LXNpemU6MTAuNXB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss c2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250 LWZhbWlseTomcXVvdDvlvq7ova/pm4Xpu5EmcXVvdDssc2Fucy1zZXJpZiI+5Y+R5Lu25Lq6PHNw YW4gbGFuZz0iRU4tVVMiPjo8L3NwYW4+PC9zcGFuPjwvYj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5 bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q75b6u6L2v6ZuF6buRJnF1b3Q7 LHNhbnMtc2VyaWYiPiBHcmVnIE1pcnNreSBbbWFpbHRvOmdyZWdpbWlyc2t5QGdtYWlsLmNvbV0N Cjxicj4NCjwvc3Bhbj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eTomcXVvdDvlvq7ova/pm4Xpu5EmcXVvdDssc2Fucy1zZXJpZiI+5Y+R6YCB5pe26Ze0PHNwYW4g bGFuZz0iRU4tVVMiPjo8L3NwYW4+PC9zcGFuPjwvYj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q75b6u6L2v6ZuF6buRJnF1b3Q7LHNh bnMtc2VyaWYiPiAyMDIxPC9zcGFuPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O+W+rui9r+mbhem7kSZxdW90OyxzYW5zLXNlcmlmIj7lubQ8c3BhbiBsYW5n PSJFTi1VUyI+Mzwvc3Bhbj7mnIg8c3BhbiBsYW5nPSJFTi1VUyI+Nzwvc3Bhbj7ml6U8c3BhbiBs YW5nPSJFTi1VUyI+DQogNDoyMDxicj4NCjwvc3Bhbj48Yj7mlLbku7bkuro8c3BhbiBsYW5nPSJF Ti1VUyI+Ojwvc3Bhbj48L2I+PHNwYW4gbGFuZz0iRU4tVVMiPiBkcmFmdC1nZW5nLTZtYW4tcmVk dW5kYW5jeS1wcm90ZWN0aW9uLXNyaEBpZXRmLm9yZzxicj4NCjwvc3Bhbj48Yj7mioTpgIE8c3Bh biBsYW5nPSJFTi1VUyI+Ojwvc3Bhbj48L2I+PHNwYW4gbGFuZz0iRU4tVVMiPiA2bWFuIFdHICZs dDtpcHY2QGlldGYub3JnJmd0OzsgRGV0TmV0IFdHICZsdDtkZXRuZXRAaWV0Zi5vcmcmZ3Q7OyBH cmVnIE1pcnNreSAmbHQ7Z3JlZ29yeS5taXJza3lAenRldHguY29tJmd0Ozxicj4NCjwvc3Bhbj48 Yj7kuLvpopg8c3BhbiBsYW5nPSJFTi1VUyI+Ojwvc3Bhbj48L2I+PHNwYW4gbGFuZz0iRU4tVVMi PiBGbG93IElkZW50aWZpY2F0aW9uIGluIElQdjY8bzpwPjwvbzpwPjwvc3Bhbj48L3NwYW4+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiPjxvOnA+Jm5ic3A7PC9v OnA+PC9zcGFuPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJF Ti1VUyI+RGVhciBBdXRob3JzLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxkaXY+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyI+dGhhbmsgeW91IGZvciBicmluZ2luZyB5 b3VyIHByb3Bvc2FsIHRvIHRoZSBkaXNjdXNzaW9uLiBJIGFncmVlIHdpdGggeW91ciB2aWV3IHRo YXQgdGhlIGV4cGxpY2l0IHJvdXRpbmcgZW5hYmxlZCBieSBTUnY2IGNyZWF0ZXMgYW4gZW52aXJv bm1lbnQgd2hlcmUgUFJFT0YgY2FuIGJlIHVzZWQuIEFuZCwgYXMgd2Uga25vdywgVGhlIFBSRU9G Jm5ic3A7bWF5IGJlIHVzZWQgaW4gYSBEZXROZXQNCiBkb21haW4gdG8gbG93ZXIgcGFja2V0IGxv c3MgcmF0aW8gYW5kIHByb3ZpZGUgYm91bmRlZCBsYXRlbmN5LiZuYnNwOzxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9 IkVOLVVTIj5BZnRlciByZWFkaW5nIHRoZSBkcmFmdCwgSSd2ZSBnb3QgYSBxdWVzdGlvbiBmb3Ig eW91LiBXaGF0IGRvIHlvdSBzZWUmbmJzcDthcyB0aGUgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSBJ UHY2IEZsb3cgTGFiZWwgcGVyIFJGQyA2NDM3IGFuZCB0aGUgRmxvdyBJZGVudGlmaWNhdGlvbiBm aWVsZCBpbiB0aGUgVExWIHByb3Bvc2VkJm5ic3A7aW4gdGhlIGRyYWZ0PyBDb3VsZCB0aGUgSVB2 NiBGbG93DQogTGFiZWwgYmUgdXNlZCB0byBpZGVudGlmeSB0aGUgZmxvdyBmb3IgdGhlIFBSRU9G PzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIGxhbmc9IkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rp dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyI+UmVnYXJk cyw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBsYW5nPSJFTi1VUyI+R3JlZzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2 Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg== --_000_3fdc1006788e47e59cfb8dcc03e9bce6huaweicom_-- From nobody Mon Mar 8 01:59:07 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5949C3A292C; Mon, 8 Mar 2021 01:59:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.247 X-Spam-Level: X-Spam-Status: No, score=-2.247 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a-0yc7bQ59Tn; Mon, 8 Mar 2021 01:59:02 -0800 (PST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2088.outbound.protection.outlook.com [40.107.20.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDD653A292A; Mon, 8 Mar 2021 01:59:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eE5+4Nz7Q/Ay1Lu+M4AueYVT7Nt3+M/UE94jcLAcHYiQCmUtccqihWwLEBe6bYRzygD0dBIAM1iPn11kkyMtXpthqxqx/786ZILLY6jSVixmP9cxz7Mh02vQ33NY32eG2G9BXuquD8cnlDYoAU1kHzpYFd+Fyo+S5mWtg2GCqVGsBi2Z3ZnHRGquQnTFQVf9KeXmJzotxTh6KpjrALRrvk5V72JmAsCrZFXvmb8id/CoaBl240nvt2M5RGUJ8miNbnmD46lCn+03DyI5REOnQF8VvZFBLQFMBG3R7tQX7XJmY4uB/BSvVStQBT/9H3Z0rO6bgqaKI/P0QYZ8W/FOiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5E2cXtXZFNyK6TKKP/hPxrNqO56GN28hJ8/BcJqhd5Y=; b=TtyGTMuJ8TNQXw/wh4Ue07r43oxAqHPX98Bq3dlmZU2b3D6D1tEpC+0VNlEbsQwG48fI8yl+1/UiU8AYkl9I7TsT2S9kE6YWqzkc6Y77L///m8RYVsoHvbWURGvqJmvaEZ7SIgD+kpcykpj0M+ndAyjYza8PHwtS5+oSrpITlOPTUOkISKsHBwhymaJU5gL3MgGW9pD/bivzOdgpXtcCIjUr8swARPX+1wwC0ahixF2xdIhbJk7zBagXzAPHAsobS1/Sy22kwWKvLqXtugGgua3PSeRKXtct2EPXBKAPqTxaNZiAZwxIfvjcboirSfWRjbprQUMRjSgblVrcW2cHng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5E2cXtXZFNyK6TKKP/hPxrNqO56GN28hJ8/BcJqhd5Y=; b=hxDfROBUI+BPkXJzjA4Y9ad5PoTrOf2ew6CqnD+fKr2NyON0UvBI/VW+NpWU/gjny0XnR6S7QpXUvfZwfCpG3LxIZOCfPdZsPt5IFu5H+2vH7Yush+mqDcJ/wCiT+P7l+6NscIyAvS4hGsNohsJUcXWzzR9qXVBFqgokxrhd3OA= Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com (2603:10a6:208:22::25) by AM4PR0701MB2132.eurprd07.prod.outlook.com (2603:10a6:200:49::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.14; Mon, 8 Mar 2021 09:58:57 +0000 Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::8d2d:fd78:9fa:5f92]) by AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::8d2d:fd78:9fa:5f92%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 09:58:56 +0000 From: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= To: Yaakov Stein , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUACYqgyAAeeRYCAAAGQfkAAGXwiQ Date: Mon, 8 Mar 2021 09:58:56 +0000 Message-ID: References: In-Reply-To: Accept-Language: hu-HU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: rad.com; dkim=none (message not signed) header.d=none;rad.com; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [185.29.82.144] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6cb9cb46-02c2-425f-c10b-08d8e218ca0c x-ms-traffictypediagnostic: AM4PR0701MB2132: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3826; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Aw8nspTqD1gj2bR5Iz3gR15fWjvXN9LHImtm20WGJtlxptHq+j/KXtl8wPRGoznWSiKgrNuISSne3MUAK71Rj/d3SX070UvFt9M/3PU9+9WvKbF2KWf2TVaOMjvk82iCZUj1RpklGrRQbre95Yz8nbEC5AaKmxmIY3tpJqwg0wJyHqGW+Uvn6f2cObyMYIgTjmZ+FrJ4fWoep3Q7E2Dq7Vb4g6+c5sgtLHvk9cwcxkCTpiym/DQIbkC/jyU7t6Ycj0dqBE743Pa8ZzIjVOzx3CUgU5RfcWgVHa04KCAfXqv0kRvCEvK48+6Iqq5D9V3gYl+CMWz5jTOs0HGVMFS0So2pP9j9H0DlBxUCy8D/CaaXr9WNhkQ8Ytih0rgqF4gjaZ1sj7VMICy3jiCI11mk/8AFJng1a7lJzPtVTY9M3rQneylmo6zVwhuLVL0fqJpBDEm5bxfYuNs5J/PsRKMi29UykOVm755mE4j0a4otjDsg973o3Eu6hyBhtyn9Hz/4WG9kSsR9Oounx3QspqXV7GMpJUK0kaIlYlVdBznxmKw+BjNT2wY+LDeVif+y965BY5xsiSWS7vCZrfALACsvrq/U2qf972ot0roxWAPd+MzzBVHhKrRbfMo5P48mYn6n/XVtrNB0H/qHUiRP4smgoQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0702MB3603.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(396003)(376002)(39860400002)(346002)(366004)(76116006)(5660300002)(86362001)(66574015)(166002)(30864003)(26005)(83380400001)(2906002)(186003)(9326002)(33656002)(8936002)(9686003)(53546011)(71200400001)(6506007)(55016002)(8676002)(7696005)(52536014)(66446008)(478600001)(64756008)(110136005)(66476007)(66946007)(66556008)(966005)(316002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?nQ5/Fol9SxNqc0yN6+s10lqrjGkjzL5X6hVVgF5pKR0ozwqL0BAyJV2RgN?= =?iso-8859-1?Q?mpWpS0NrBCVbAvq074FdutHj12XIf4Iik0d7PnkGtaLNhCLBnCbOPJzHB7?= =?iso-8859-1?Q?vKOiUplG7Az90CTzCToEDuqu4AZNrZy9AXK3t1TzfR+hzk2iMCUd+3rM3j?= =?iso-8859-1?Q?1lo2KXPJJOJUfug+X3sDlT9ZaEg9dETSO/KhP9gkYVMvNoYfBCHIbFvGjZ?= =?iso-8859-1?Q?tHjtDcOjSRAy9Zbx5ac4e4UYJIwUaSuS6EP2/TZnpassf84HLQgOKBdRuH?= =?iso-8859-1?Q?RDest00UBcqBgOLctyJKt+abbTGWfMUHaFsPLUB4qhGv7hE5+Ld5j8q8ON?= =?iso-8859-1?Q?/mHm2lSg4Zxa5SjogaQYx043I+VtxDrK4QRCoFvXY5BAZnOqloUj50yEsQ?= =?iso-8859-1?Q?HB/Ee1mqsAp5IqTr8slEyoXpbk+DASZB6FL2/OmkJhUSl/Ew+ACSI2qXbg?= =?iso-8859-1?Q?zHph3+ei3DS3G4vPkB3572qdUC9zskC6tFxFr6dK13CMWmf4IVqhReydN3?= =?iso-8859-1?Q?X5xLa8+bEAkzpHhzPxDGBgHsY5ARBW/2Cz8DrNYh2KucGqv0/CyXxIIwOE?= =?iso-8859-1?Q?GYwLH8fY5H+khp29QSNolQpz0mS/bXCyrDrM9nWlyo26gvJaCrDIOrdLl8?= =?iso-8859-1?Q?HNyQUfUVevcBgba62HiJzhhvfIkOQVBQBnRJ0oqOgjU1J6JZApJgrGedPJ?= =?iso-8859-1?Q?V5Tn39VEpxymOJenHaAp9PI4zRRRg01zYYhQjQxljoFqiKTGy6RTVHhruH?= =?iso-8859-1?Q?sEYeGLKbHER9rWmAYO79AArUqpqKuAz+4euX8VDkChf5iCnamp9hkZljSL?= =?iso-8859-1?Q?XBwtM4U82TwfyYWkSEPSk2BFRvloF2+3j7Wr56hTarDBeuTQA+iHt6bG5f?= =?iso-8859-1?Q?dv8S4EILQK43CLcKArwVrOjnkoCmvVlrRbVfX35WHxXZSpNXyIbSB1sOX4?= =?iso-8859-1?Q?na4QCjcSPwk/lNUqb9g5fgJ9l1qnD98MZogym+eO5Ro5ZTCIjEmfIZAvNJ?= =?iso-8859-1?Q?swP0m6qm7eJJS1l0ECe25GYnDKysO7wRSOFESRcO2vOAp+c1H744lmzLFo?= =?iso-8859-1?Q?1n7IJmJnM2z8EWfroUSLSknLKUIBNgM++zjjZ6hsWDRA5xZ0oEA2c0kN/S?= =?iso-8859-1?Q?CU7osOWinwerRMr4zSosb9qdt7WR+5FXqzEVHRixa3xzzPa9B7Yzn8yonG?= =?iso-8859-1?Q?T/cpTrV1lw79ztEVgjWfAJKcoHKHWCcEwVOa3My6Zw8/aE5gSuqzmQrHYx?= =?iso-8859-1?Q?aekrWXuuMtoifbjO45sXo3xRRcmV9jin8InBK7pewbaqEqwyEsM83Ig5Ry?= =?iso-8859-1?Q?mBvCNH2Qs6CIZAPD55AnupxQ9zgcxgQ0Ngplyfk9dlOTun24Kl37phIfWS?= =?iso-8859-1?Q?NTBwdbNAO8?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR0702MB3603D780FAD4BA967CC89D08AC939AM0PR0702MB3603_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR0702MB3603.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6cb9cb46-02c2-425f-c10b-08d8e218ca0c X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 09:58:56.8611 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cgLkuaAoW4V+46wsR8Edpmvbhipf3K1RJmxP2KVDzKjEJ/X1dW71sxezIGCxYuK2222GiUgTMFGUVm835wxw+JKK94RITk1pt7hg7iN4DZA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0701MB2132 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 09:59:06 -0000 --_000_AM0PR0702MB3603D780FAD4BA967CC89D08AC939AM0PR0702MB3603_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Yaakov, Thanks for the clarifications, that helps a lot to position correctly your = draft. As You have wrote: "Determinism is not the main goal of this draft" what was not clear from the text. I may be somewhat mis-leaded by the name You have used for the concept. TSN is widely used in IEEE as "Time-Sensitive Networking" and its main targ= et is deterministic communication. You may consider to use a different abbreviation to avoid that confusion. E= .g., "srtsc" (segment routing time sensitive communication) or anything else You= like. Many thanks Bala'zs From: Yaakov Stein Sent: Monday, March 8, 2021 8:27 AM To: Bal=E1zs Varga A ; detnet@ietf.org; spring= @ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Bala'zs, Sorry that I never got back to you on this. It is much easier to shoot back replies to shorter emails :) > 1, What is the node behavior if deadtime expires? Nothing special is done if a local deadline is exceeded. There is still hop= e that the time can be made up. If too many packets fail to conform to their overall budget, then maybe something has changed and reoptimization (or realizing that the = application budget is not feasible) is required. > 2, Do I understand it correctly that the described solution is practicall= y a new per-hop-behavior? Well, new to here but hardly "new" (similar things were done in ATM...) The new per-router behavior requires a new data structure in the router ins= tead of a FIFO queue, but there is no absolute requirement for other tools. As I just hinted and have mentioned in a previous email, later on we may ne= ed some OAM to see if configuration needs to be updated. I have already experimented with periodic RSVP-like synthetic packets that = collect timestamps along the path. > 3, Design of a guaranteed upper bound usually requires deterministic arri= val at each node along the path. Here is where we differ (or at least put different emphasis on the word "us= ually". Determinism is not the main goal of this draft. Meeting challenging delay b= udgets is. Sometimes absolute determinacy is required (e.g., in some forms of "telepro= tection" in some electricity distribution networks) in which case either TDM/Qbv/calendaring can be used, or this method couple= d with a differential delay compensation buffer. I agree that PDV increases from hop to hop to an extent, but when the delay= budget is really challenging you simply can't allow that many hops. In mixed 5G xHaul cases in which I have been involved there are no more tha= n 4 hops (usually fewer). If you wish something more deterministic then look at the Andrews-Zhang str= ategy. They gate at the ingress router and EDF at all the following ones. They show that (statistically) they ride "green waves" and thus the PDV doe= sn't increase significantly (see their graphs). This comes at the expense of a long cycle time for the = initial gate, which means the budget can't be too tight. > 4, Wire-speed timestamping and calculation I come from the PTP world where hardware timestamping with submicro resolut= ion is common. Since the timestamp is on the rising edge of the first bit, i.e., before th= e classifier, it is carried out and placed in the packet metadata for every packet whethe= r or not it is PTP. I always shed tears over throwing out this information. The RSVP-like method just mentioned simply copies it into the packet. The SRTSN packets use this information in the scheduler. I have consulted my ASIC people about the use of priority heaps and insert-= queues and they ensure me that they can put something like that into our 100G chip= . > 5, Time gated Queuing [802.1Qbv] I didn't say that in principle you can't build a Qbv system that does great= things. It is flexible enough to do anything you want. The problem is the scalability. I have played around with optimizing it and haven't found a good "on line" = algorithm, that is, a way of adding a new flow to an existing optimized design. And even in the "off line" case (clean slate with all flows known) slight c= hanges and even bad clocks in the user equipment ruins things. But the main idea of this draft is that even given a good on-line algorithm= for Qbv you would need to distribute the changes in the all the schedules to all th= e routers. What happens in the meantime? What happens when one router is already updat= ed but another isn't? With the stack method only a single ingress router needs to be updated and = consistency is ensured! > 6, Deadtime calculation is also complex if impact of other DetNet flows m= ust be considered. Of course! The offset vector needs to be optimized. I have several such methods and am running simulations of various strategie= s. As stated in the ID the specific calculation was for illustration purposes = - the stack mechanism is much more generic. > 7, Impact of some DetNet functions I have not yet gotten to the point where I have considered this. > 8, The ever-green topic ( and not the green-wave :--))) ): label stack s= ize. Yes, I didn't want to bring this up yet, but have been forced to. I have to admit that I am not a fan of SRv6 for just this reason. But, the SRTSN stack can be really small! SR stack entries need only be ceil ( log2 (number of routers) ) in size (re= quires a little local information base in the router) assuming once the stack is popped you use the original DA. If you are not w= illing to use this additional information you still only need sufficient suffixes, which is still small. The deadline entries can be miniscule since the wraparound constraint is de= rived from the maximum flight time of a packet in the network. In my designs I have gotten away with 16-32 bits per entry, which means 4-8= hop stacks only require the room of a single IPv6 address. Hope this answers your questions and kicks off a fruitful discussion. Y(J)S From: Bal=E1zs Varga A > Sent: 08/03/2021 08:35 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Any view on these comments? Thanks Bala'zs From: detnet > On B= ehalf Of Bal=E1zs Varga A Sent: Friday, February 26, 2021 3:03 PM To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Hi Yaakov, many thanks for this well-written and information-rich draft. I have enjoyed to read it (and also the discussions on the lists). :--))) Conceptual (clarification) questions: 1, What is the node behavior if deadtime expires? Drop? That requires a qui= te good design/allocation of the latency budget. Otherwise it may result in= unnecessary loss of a packet being relative late at a hop, however that mi= ght be compensated by the remaining hops. (E.g., see your Figure 2, getting= 35 microsecond at R1 instead of the designed 25, but R2,R3,R4 adds only 10= -10-10 extra microseconds). You may consider to add a "late flag" to the pa= cket instead of a drop ... 2, Do I understand it correctly that the described solution is practically = a new per-hop-behavior? Does it have no tools to "influence" the arrival ti= me at the next hops? In end-to-end latency design the DetNet/TSN functions = are often used to "adapt" the arrival of packets belonging to a DetNet flow= /TSN stream at given network points. If "latency design" requires such a "t= ime shift" then the described method has to be combined with other DetNet/T= SN tools. 3, Design of a guaranteed upper bound usually requires deterministic arriva= l at each node along the path. In this solution the flow becomes less-an-le= ss deterministic as it reaches the destination. I mean e.g., as per Figure = 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-1= 20 usec and at R4 is 92-167 usec. So, the flow is getting less-and-less det= erministic at each hop. That can make deterministic design of other flows p= assing R4 more difficult or even impossible in some scenarios. Have I misse= d something here? 4, Wire-speed timestamping and calculation: I would be interested in your v= iew regarding how realistic are (1) the wire speed timestamping capability = and (2) adding multiple calculated deadline to packets. Both are required f= or this solution. DetNet flows can have high bandwidth, not like 1588 packe= ts. Specific comments (Section 2-3): 5, Time gated Queuing [802.1Qbv] allows multiple gates to be open simultane= ously. Furthermore preemption [802.1Qbu] can be combined with Qbv. So I can= not fully agree with your evaluation of time-aware scheduling. (Section 2, = "However, time-aware scheduling suffers from two major disadvantages.") Of = course with a bad design one may shoot in his/her own leg, but in well-desi= gned scenarios efficiency loss can be eliminated without expensive computat= ion. 6, Deadtime calculation is also complex if impact of other DetNet flows mus= t be considered. The pre-calculation of individual "local" deadlines may ne= ed to be re-calculated when flows added to the network and they are using s= ome common links. So I cannot agree with your statement. (Section 3, "... S= ince the ingress router inserts the deadline stack into the packet headers,= no other router needs to be aware of the requirements of the time sensitiv= e flows. Hence admitting a new flow only requires updating the information= base of the ingress router."). Adding a flow may result in updating many i= ngress routers' configuration. Some topics for further discussions/considerations: 7, Impact of some DetNet functions (e.g., PREOF [RFC8655]) needs further di= scussions. The usage of replication/elimination impacts the deadline calcul= ation. Disjoint paths used by member flows requires different label stack a= nd deadlines. Furthermore, these path specific labels+deadlines must be add= ed by the replication node (PRF). So at least, a combination of PRF with B-= SID and computing/adding deadlines (ingress SRTSN function) seems to be nec= essary in your solution. 8, The ever-green topic ( and not the green-wave :--))) ): label stack size= . You need multiple labels per hop and the label stack contains them for ea= ch hop. Could result in a quite big label stack to be pushed at the ingress= (nx10s of labels). I am happy to have further discussions on this interesting idea Thanks & Cheers Bala'zs From: detnet > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 2:14 PM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [Detnet] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR0702MB3603D780FAD4BA967CC89D08AC939AM0PR0702MB3603_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Yaakov,

Thanks for the clarifications, that helps a lot to p= osition correctly your draft.

As You have wrote: “Determinism is not the mai= n goal of this draft” what

was not clear from the text.

I may be somewhat mis-leaded by the name You have us= ed for the concept.

TSN is widely used in IEEE as “Time-Sensitive = Networking” and its main target

is deterministic communication.

You may consider to use a different abbreviation to = avoid that confusion. E.g.,

“srtsc” (segment routing time sensitive = communication) or anything else You like.

Many thanks

Bala’zs

From: Yaakov Stein <yaakov_s@rad.com> <= br> Sent: Monday, March 8, 2021 8:27 AM
To: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>; detnet@iet= f.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Bala'zs,

Sorry that I never got back to you on this.

It is much easier to shoot back replies to shorter e= mails :)

> 1, What is the node behavior if deadtime expire= s?

Nothing special is done if a local deadline is excee= ded. There is still hope that the time can be made up.

If too many packets fail to conform to their overall= budget,

then maybe something has changed and reoptimization = (or realizing that the application budget is not feasible) is required.

> 2, Do I understand it correctly that the descri= bed solution is practically a new per-hop-behavior?

Well, new to here but hardly “new” (simi= lar things were done in ATM…)

The new per-router behavior requires a new data stru= cture in the router instead of a FIFO queue,

but there is no absolute requirement for other tools= .

As I just hinted and have mentioned in a previous em= ail, later on we may need some OAM to see if configuration needs to be upda= ted.

I have already experimented with periodic RSVP-like = synthetic packets that collect timestamps along the path.

> 3, Design of a guaranteed upper bound usually r= equires deterministic arrival at each node along the path.

Here is where we differ (or at least put different e= mphasis on the word “usually”.

Determinism is not the main goal of this draft. Meet= ing challenging delay budgets is.

Sometimes absolute determinacy is required (e.g., in= some forms of “teleprotection” in some electricity distributio= n networks)

in which case either TDM/Qbv/calendaring can be used= , or this method coupled with a differential delay compensation buffer.

I agree that PDV increases from hop to hop to an ext= ent, but when the delay budget is really challenging you simply can’t= allow that many hops.

In mixed 5G xHaul cases in which I have been involve= d there are no more than 4 hops (usually fewer).

If you wish something more deterministic then look a= t the Andrews-Zhang strategy.

They gate at the ingress router and EDF at all the f= ollowing ones.

They show that (statistically) they ride “gree= n waves” and thus the PDV doesn’t increase significantly

(see their graphs). This comes at the expense of a l= ong cycle time for the initial gate, which means the budget can’t be = too tight.

> 4, Wire-speed timestamping and calculation=

I come from the PTP world where hardware timestampin= g with submicro resolution is common.

Since the timestamp is on the rising edge of the fir= st bit, i.e., before the classifier,

it is carried out and placed in the packet metadata = for every packet whether or not it is PTP.

I always shed tears over throwing out this informati= on.

The RSVP-like method just mentioned simply copies it= into the packet.

The SRTSN packets use this information in the schedu= ler.

I have consulted my ASIC people about the use of pri= ority heaps and insert-queues

and they ensure me that they can put something like = that into our 100G chip.

> 5, Time gated Queuing [802.1Qbv]

I didn’t say that in principle you can’t= build a Qbv system that does great things.

It is flexible enough to do anything you want.<= /o:p>

The problem is the scalability.

I have played around with optimizing it and haven= 217;t found a good “on line” algorithm,

that is, a way of adding a new flow to an existing o= ptimized design.

And even in the “off line” case (clean s= late with all flows known) slight changes and even bad clocks in the user e= quipment ruins things.

But the main idea of this draft is that even given a= good on-line algorithm for Qbv

you would need to distribute the changes in the all = the schedules to all the routers.

What happens in the meantime? What happens when one = router is already updated but another isn’t?

With the stack method only a single ingress router n= eeds to be updated and consistency is ensured!

> 6, Deadtime calculation is also complex if impa= ct of other DetNet flows must be considered.

Of course! The offset vector needs to be optimized.<= o:p>

I have several such methods and am running simulatio= ns of various strategies.

As stated in the ID the specific calculation was for= illustration purposes –

the stack mechanism is much more generic.=

> 7, Impact of some DetNet functions
I have not yet gotten to the point where I have cons= idered this.

> 8, The ever-green topic ( and not the gre= en-wave :--))) ): label stack size.

Yes, I didn’t want to bring this up yet, but h= ave been forced to.

I have to admit that I am not a fan of SRv6 for just= this reason.

But, the SRTSN stack can be really small!=

SR stack entries need only be ceil ( log2 (number of= routers) ) in size (requires a little local information base in the router= )

assuming once the stack is popped you use the origin= al DA. If you are not willing to use this additional information

you still only need sufficient suffixes, which is st= ill small.

The deadline entries can be miniscule since the wrap= around constraint is derived from the maximum flight time of a packet in th= e network.

In my designs I have gotten away with 16-32 bits per= entry, which means 4-8 hop stacks only require the room of a single IPv6 a= ddress.

Hope this answers your questions and kicks off a fru= itful discussion.

Y(J)S

From: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>
Sent: 08/03/2021 08:35
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Any view on these comments?

Thanks

Bala’zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Bal=E1zs Varga A
Sent: Friday, February 26, 2021 3:03 PM
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

many thanks for this well-written and information-ri= ch draft.

I have enjoyed to read it (and also the discussions = on the lists). :--)))

Conceptual (clarification) questions:

1, What is the node behavior if deadtime expires? Dr= op? That requires a quite good design/allocation of the latency budget. Oth= erwise it may result in unnecessary loss of a packet being relative late at= a hop, however that might be compensated by the remaining hops. (E.g., see your Figure 2, getting 35 microsecond at= R1 instead of the designed 25, but R2,R3,R4 adds only 10-10-10 extra micro= seconds). You may consider to add a "late flag" to the packet ins= tead of a drop ...

2, Do I understand it correctly that the described s= olution is practically a new per-hop-behavior? Does it have no tools to &qu= ot;influence" the arrival time at the next hops? In end-to-end latency= design the DetNet/TSN functions are often used to "adapt" the arrival of packets belonging to a DetNet flow/TSN= stream at given network points. If "latency design" requires suc= h a "time shift" then the described method has to be combined wit= h other DetNet/TSN tools.

3, Design of a guaranteed upper bound usually requir= es deterministic arrival at each node along the path. In this solution the = flow becomes less-an-less deterministic as it reaches the destination. I me= an e.g., as per Figure 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-120 usec an= d at R4 is 92-167 usec. So, the flow is getting less-and-less deterministic= at each hop. That can make deterministic design of other flows passing R4 = more difficult or even impossible in some scenarios. Have I missed something here?

4, Wire-speed timestamping and calculation: I would = be interested in your view regarding how realistic are (1) the wire speed t= imestamping capability and (2) adding multiple calculated deadline to packe= ts. Both are required for this solution. DetNet flows can have high bandwidth, not like 1588 packets.

Specific comments (Section 2-3):
5, Time gated Queuing [802.1Qbv] allows multiple gat= es to be open simultaneously. Furthermore preemption [802.1Qbu] can be comb= ined with Qbv. So I cannot fully agree with your evaluation of time-aware s= cheduling. (Section 2, "However, time-aware scheduling suffers from two major disadvantages.") Of course with a b= ad design one may shoot in his/her own leg, but in well-designed scenarios = efficiency loss can be eliminated without expensive computation.=

6, Deadtime calculation is also complex if impact of= other DetNet flows must be considered. The pre-calculation of individual &= quot;local" deadlines may need to be re-calculated when flows added to= the network and they are using some common links. So I cannot agree with your statement. (Section 3, "... Since = the ingress router inserts the deadline stack into the packet headers, no o= ther router needs to be aware of the requirements of the time sensitive flo= ws. Hence admitting a new flow only requires updating the information base of the ingress router."). Addi= ng a flow may result in updating many ingress routers' configuration.<= /o:p>

Some topics for further discussions/consideration= s:

7, Impact of some DetNet functions (e.g., PREOF [RFC= 8655]) needs further discussions. The usage of replication/elimination impa= cts the deadline calculation. Disjoint paths used by member flows requires = different label stack and deadlines. Furthermore, these path specific labels+deadlines must be added by the rep= lication node (PRF). So at least, a combination of PRF with B-SID and compu= ting/adding deadlines (ingress SRTSN function) seems to be necessary in you= r solution.

8, The ever-green topic ( and not the green-wave :--= ))) ): label stack size. You need multiple labels per hop and the label sta= ck contains them for each hop. Could result in a quite big label stack to b= e pushed at the ingress (nx10s of labels).

I am happy to have further discussions on this inter= esting idea

Thanks & Cheers

Bala'zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 2:14 PM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [Detnet] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR0702MB3603D780FAD4BA967CC89D08AC939AM0PR0702MB3603_-- From nobody Mon Mar 8 03:45:34 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2472D3A0C03; Mon, 8 Mar 2021 03:45:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.799 X-Spam-Level: X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K2PWawMwH-8Z; Mon, 8 Mar 2021 03:45:24 -0800 (PST) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30043.outbound.protection.outlook.com [40.107.3.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9EF9D3A0C01; Mon, 8 Mar 2021 03:45:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CKktFyNbf7Rg685J2RpbyLUogzKjKVxfC6bpFdipWcL6o8kxJcNxRwBNEoE9csecNtgudTWZiG67ZflKI08ZtAhWSISNvzG1R3w9AXyOMaV3UNTHF3lqHfIEdGxCgOmbGLnyVQlO0CnLJ77C6AOJ+55ISNPI0xQWoa2/ZrjvkK8jaf6nK+E0PPtSAHTdNgKDdW5u748YkQak30b0LdY9jGmWQYWRRz9CGJjZz+dB/mcepKyqFfK5rKQY2taiFGTXg9J/R1L+nXj9teAtZSrGyKm3Egy2etiI0MN4P+OlsG6fTMc/6T1GNGVdoGQRWjR5m0Uf6PRphnwKIH2b/p2hmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lr93Zn5KF5Zyo8cEcX59h630M9Cjr140u0Q2GAqX3qg=; b=bMSaUPURsUbU1s9SBl0pGO86mvP412zW2glYmb2Tn5ghIe5qNGVx5UmG2/F3ac6GtcdfxtJta2lHT3upfix6uN2j2CMabc22oeLn57FhCzBfXUj8ABIh/dHrXWb5nDOag13sfZTyZR1dlKkgmMeVYEnPW3ixeHelKJnE4Sksw6LrBn7HL5GB5ERxpcqDkO2KU/7n50+YnsR9Ppxtms2YnwTFsd4++xiK/yuwhVxEBAuwpGRfykT6zLsceccomZc6YWeKU1AYK4Eou2HICZSdzIUSL02+7y7GhlDTkvCcUZjt5+Wu90yMciOuXl06bMKHBK/CTd8ruHct3aETBkd8Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lr93Zn5KF5Zyo8cEcX59h630M9Cjr140u0Q2GAqX3qg=; b=SWnAQsqSrDwpEFpIA2I7IHM/6e3e9A0xO3UScpcUV5PW6QbcW656xPTwRr8unhslMGOOsdd8BFp1bSz2gOaaBGR+sdCrFnFL6IgFuJNdHUzIlNTgi+e2TWGRjHzowzPuPsEuO58T2Lq8J+eoshhicJl4qgEoZfPueqp4lzPP1rk= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM0PR03MB5492.eurprd03.prod.outlook.com (2603:10a6:208:173::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.20; Mon, 8 Mar 2021 11:45:20 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 11:45:20 +0000 From: Yaakov Stein To: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUACYqgyAAeeRYCAAAGQfkAAGXwiQAAPETvA= Date: Mon, 8 Mar 2021 11:45:20 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ericsson.com; dkim=none (message not signed) header.d=none;ericsson.com; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 568fc3b9-d03e-4693-4c26-08d8e227a6f3 x-ms-traffictypediagnostic: AM0PR03MB5492: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: NhWHFh5xPhwGi5L8QI0fCCm0GpyNC/l3t+JB/xzJeVXoaoWZpcc6yej9C1i+4KjNfquaxeKXb15Qn5hQvXZfUO6Ova+hw+STGxP7EEDepccDtT1PdAhegmskotWLWkKH/+dyDqCE38e3km2aD+5IsZ8Ycim7d/075HZ9pdMUsxeS42s84fmq566o0I/eqQEGQGpFN/s6gy2oN4HjpaKH/J2Y9Te6RlVVeMZ5WCP+Xg2ioi+DzrihY2SdGkjPckgXdExIhZrAIi2CB9hyNt/xwaFuvQRW+Zu0FZCiHJ8Rc+HuDfAS/uRKtcly6qWJOyrAdNDMxter9DQbkLxAd93dhqXWUaBHLJzOtGhVQQloNu8cR6T9cAeUzKLCXIj0cCje9K1x4ImOTRd1EBrNJxFFL8fEsP8X8XZoxFub9T5F01CcrseMm2sU+a0SBbHZBWYLeG/Wwe/b83V9aaS6gTpj2mkvdfsOzvTzjDr4KYjU6Ag3wZTvS6I/QjPk9Ow9BHB/zmRea5/IaTBamSPm2wJH4obK2F1M9NZjgD82vvwmxfdHPVo7l4i9sYEtxDAvVZmp+ODpIrkxVU63/GoTusYI43Fkko2mR1qjkM3++Xqgndo= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39850400004)(376002)(346002)(366004)(136003)(8936002)(52536014)(7696005)(186003)(55016002)(9686003)(966005)(26005)(66446008)(6506007)(66476007)(64756008)(53546011)(66556008)(66946007)(83380400001)(76116006)(478600001)(33656002)(86362001)(66574015)(166002)(5660300002)(110136005)(30864003)(8676002)(2906002)(71200400001)(316002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?nAxfUe4YCz0CpEquA0jZA3cHJ+KMSaYQx7WbFR5e9US9dcF6U8d5gKk1Lh?= =?iso-8859-1?Q?KpMoIfPL4olS65UpKT+DVELiPGRGyoToFM1ZrhAr10AbTYge/qya9mIwqR?= =?iso-8859-1?Q?NJCtnFk5tC9cdka3r0vxCxt8AOZTwsFCqGcF2URw7FeXiOcYMXk8T5eI8k?= =?iso-8859-1?Q?uo7x4A9V5OsXQ0nj14xJRyrGMvZWNsslGYv/4sYHOmqu8bfO17hkZagv/K?= =?iso-8859-1?Q?T8r6QcgkOYLwoRPrgWoqctOvmRLiUqf7uB3WsaD35fVpCedFAWlzTrq3ew?= =?iso-8859-1?Q?8rqnxcOZv3mqA/bW/vFc3OW+xBm5Gtm9snSYuKsFnHuKs6Q/yvCkhtUB62?= =?iso-8859-1?Q?C6dPTwWq1sObSPJUPKs+GZeB2PMuUFrOTHEvVVogZ/CSQypoS/TY8Ykd0p?= =?iso-8859-1?Q?xtESDYq8MRk5mb9JoVCPUBq5WOUGZsxyHBjsKZ1qNv7RwftJQAPoo5egAc?= =?iso-8859-1?Q?j98igdu/lESjK1+mXvTWNN6xRR78tuAnQ0C591TujVbyDnbvWotVTjliHa?= =?iso-8859-1?Q?25mAfbvWHitk274oAs+OFI3EfXhJwmW+XTzGKg4kCdPDXNc4BDocoS73D1?= =?iso-8859-1?Q?wTQljfJ2acFfPzl3lxdoXuRy5oDdSMGfvuTnsH4Iucu959y4AuGI/h+i/l?= =?iso-8859-1?Q?4z911XhYn/3AyhH2IkDz8BRJlY1BMu6qefNhA+K5T9/nNJz1dnMtnDS3do?= =?iso-8859-1?Q?qZTfkUtee5UY+QfBARF0ZvCae+UUgKUhDBIJE+WNBWijRomn4EkYsGgoFE?= =?iso-8859-1?Q?8SQkRs6IJZ/3bPKN4YQDjrmYSfR4luws2rQL8FGKTRQJgPG/o4g/Mwsphj?= =?iso-8859-1?Q?RhOPZRplSc6E/HJ2djJpSt23vajIUUby6qIOgQa9YD2TqZkhci5IlOOG5E?= =?iso-8859-1?Q?cckrRMkmoHvYnIarfYUUVRWtkw1nNRu9I2SLiFrqyZn9fo2cW7mUO26Fhb?= =?iso-8859-1?Q?VM4iaVBOCQ4Khm1kBDbn63VkbEMFsWg4rraDKZvbeBKqaZU5VJbRcRQzlB?= =?iso-8859-1?Q?libp7uQIeQxSCrUOkRAMA3BHfjWi+GyiwKrPM+Egpg7YS7E/PqOda5eF6Q?= =?iso-8859-1?Q?bt7o/mgnntMhHl00Bfq4pY6lRzXPLoPx7ibaN/U9M71yPZ/xSNZNJwWf5X?= =?iso-8859-1?Q?QvdJPz4gh2QEH0Wn0dFT12fUMX1FJl1Ow3Ld5nLVd5PMgHhXD+8aXbUyyY?= =?iso-8859-1?Q?cn9xiCf0+0H6MmqMn37fdVehQo4kMj8Nd+isTEU809pKGpWV95WgLFMum4?= =?iso-8859-1?Q?lQJu7RZqvf0KpsPpSzotpe2bRiLl5cHO2AsSa6vlkP0BsM2DyvDlMo1w03?= =?iso-8859-1?Q?VfeAyXt2/okCBZkPxwESY/R5xPKvrp+OhTN5Ul/1LIxwiG3zyIz3U5mZn3?= =?iso-8859-1?Q?j5Hj5RDGXq?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR03MB35224845605B31C5C53198D4E5939AM0PR03MB3522eurp_" MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 568fc3b9-d03e-4693-4c26-08d8e227a6f3 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 11:45:20.4890 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /bFk68j6d621s73AR3fSlTzjSf7RpHT7HJmUJAFfs+XkBZusS16008noRqk/YYQdA6ehrm5iXdRlZy0ic6/egg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5492 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 11:45:28 -0000 --_000_AM0PR03MB35224845605B31C5C53198D4E5939AM0PR03MB3522eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Bala'zs, The IEEE definition of TSN mentions determinism, but defines it not as cons= tant delay as in a TDM service, but rather as to provide deterministic services through IEEE 802 networks, i.e., guaranteed packet transport with bounded latency, low packet delay va= riation, and low packet loss. So "determinism" in this context means bounded latency, low PDV and low PLR= . DetNet is aligned with this definition. From the charter: deterministic data paths that operate over Layer 2 bridged and Layer 3 rout= ed segments, where such paths can provide bounds on latency, loss, and pack= et delay variation (jitter), and high reliability. So it is the path that is deterministic, but the packet parameters need to = be bounded. In Carrier grade Ethernet "determinism" is usually taken to mean that a flo= w is pinned to a path, that is, we know precisely which network elements a packet will traverse, although delay, PLR, etc. are parameters to be set and/or measured. When I say that a packet or flow is "time sensitive" I merely mean that the= re is a time budget to be attained. However, SRTSN by any other name smells as sweet to me. Stack-based Time-sensitive Enhancements for Immediate-delivery Networking (= STEIN) could also work :) . Y(J)S From: Bal=E1zs Varga A Sent: 08/03/2021 11:59 To: Yaakov Stein ; detnet@ietf.org; spring@ietf.org; pce@= ietf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Thanks for the clarifications, that helps a lot to position correctly your = draft. As You have wrote: "Determinism is not the main goal of this draft" what was not clear from the text. I may be somewhat mis-leaded by the name You have used for the concept. TSN is widely used in IEEE as "Time-Sensitive Networking" and its main targ= et is deterministic communication. You may consider to use a different abbreviation to avoid that confusion. E= .g., "srtsc" (segment routing time sensitive communication) or anything else You= like. Many thanks Bala'zs From: Yaakov Stein > Sent: Monday, March 8, 2021 8:27 AM To: Bal=E1zs Varga A >; detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Bala'zs, Sorry that I never got back to you on this. It is much easier to shoot back replies to shorter emails :) > 1, What is the node behavior if deadtime expires? Nothing special is done if a local deadline is exceeded. There is still hop= e that the time can be made up. If too many packets fail to conform to their overall budget, then maybe something has changed and reoptimization (or realizing that the = application budget is not feasible) is required. > 2, Do I understand it correctly that the described solution is practicall= y a new per-hop-behavior? Well, new to here but hardly "new" (similar things were done in ATM...) The new per-router behavior requires a new data structure in the router ins= tead of a FIFO queue, but there is no absolute requirement for other tools. As I just hinted and have mentioned in a previous email, later on we may ne= ed some OAM to see if configuration needs to be updated. I have already experimented with periodic RSVP-like synthetic packets that = collect timestamps along the path. > 3, Design of a guaranteed upper bound usually requires deterministic arri= val at each node along the path. Here is where we differ (or at least put different emphasis on the word "us= ually". Determinism is not the main goal of this draft. Meeting challenging delay b= udgets is. Sometimes absolute determinacy is required (e.g., in some forms of "telepro= tection" in some electricity distribution networks) in which case either TDM/Qbv/calendaring can be used, or this method couple= d with a differential delay compensation buffer. I agree that PDV increases from hop to hop to an extent, but when the delay= budget is really challenging you simply can't allow that many hops. In mixed 5G xHaul cases in which I have been involved there are no more tha= n 4 hops (usually fewer). If you wish something more deterministic then look at the Andrews-Zhang str= ategy. They gate at the ingress router and EDF at all the following ones. They show that (statistically) they ride "green waves" and thus the PDV doe= sn't increase significantly (see their graphs). This comes at the expense of a long cycle time for the = initial gate, which means the budget can't be too tight. > 4, Wire-speed timestamping and calculation I come from the PTP world where hardware timestamping with submicro resolut= ion is common. Since the timestamp is on the rising edge of the first bit, i.e., before th= e classifier, it is carried out and placed in the packet metadata for every packet whethe= r or not it is PTP. I always shed tears over throwing out this information. The RSVP-like method just mentioned simply copies it into the packet. The SRTSN packets use this information in the scheduler. I have consulted my ASIC people about the use of priority heaps and insert-= queues and they ensure me that they can put something like that into our 100G chip= . > 5, Time gated Queuing [802.1Qbv] I didn't say that in principle you can't build a Qbv system that does great= things. It is flexible enough to do anything you want. The problem is the scalability. I have played around with optimizing it and haven't found a good "on line" = algorithm, that is, a way of adding a new flow to an existing optimized design. And even in the "off line" case (clean slate with all flows known) slight c= hanges and even bad clocks in the user equipment ruins things. But the main idea of this draft is that even given a good on-line algorithm= for Qbv you would need to distribute the changes in the all the schedules to all th= e routers. What happens in the meantime? What happens when one router is already updat= ed but another isn't? With the stack method only a single ingress router needs to be updated and = consistency is ensured! > 6, Deadtime calculation is also complex if impact of other DetNet flows m= ust be considered. Of course! The offset vector needs to be optimized. I have several such methods and am running simulations of various strategie= s. As stated in the ID the specific calculation was for illustration purposes = - the stack mechanism is much more generic. > 7, Impact of some DetNet functions I have not yet gotten to the point where I have considered this. > 8, The ever-green topic ( and not the green-wave :--))) ): label stack s= ize. Yes, I didn't want to bring this up yet, but have been forced to. I have to admit that I am not a fan of SRv6 for just this reason. But, the SRTSN stack can be really small! SR stack entries need only be ceil ( log2 (number of routers) ) in size (re= quires a little local information base in the router) assuming once the stack is popped you use the original DA. If you are not w= illing to use this additional information you still only need sufficient suffixes, which is still small. The deadline entries can be miniscule since the wraparound constraint is de= rived from the maximum flight time of a packet in the network. In my designs I have gotten away with 16-32 bits per entry, which means 4-8= hop stacks only require the room of a single IPv6 address. Hope this answers your questions and kicks off a fruitful discussion. Y(J)S From: Bal=E1zs Varga A > Sent: 08/03/2021 08:35 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Any view on these comments? Thanks Bala'zs From: detnet > On B= ehalf Of Bal=E1zs Varga A Sent: Friday, February 26, 2021 3:03 PM To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Hi Yaakov, many thanks for this well-written and information-rich draft. I have enjoyed to read it (and also the discussions on the lists). :--))) Conceptual (clarification) questions: 1, What is the node behavior if deadtime expires? Drop? That requires a qui= te good design/allocation of the latency budget. Otherwise it may result in= unnecessary loss of a packet being relative late at a hop, however that mi= ght be compensated by the remaining hops. (E.g., see your Figure 2, getting= 35 microsecond at R1 instead of the designed 25, but R2,R3,R4 adds only 10= -10-10 extra microseconds). You may consider to add a "late flag" to the pa= cket instead of a drop ... 2, Do I understand it correctly that the described solution is practically = a new per-hop-behavior? Does it have no tools to "influence" the arrival ti= me at the next hops? In end-to-end latency design the DetNet/TSN functions = are often used to "adapt" the arrival of packets belonging to a DetNet flow= /TSN stream at given network points. If "latency design" requires such a "t= ime shift" then the described method has to be combined with other DetNet/T= SN tools. 3, Design of a guaranteed upper bound usually requires deterministic arriva= l at each node along the path. In this solution the flow becomes less-an-le= ss deterministic as it reaches the destination. I mean e.g., as per Figure = 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-1= 20 usec and at R4 is 92-167 usec. So, the flow is getting less-and-less det= erministic at each hop. That can make deterministic design of other flows p= assing R4 more difficult or even impossible in some scenarios. Have I misse= d something here? 4, Wire-speed timestamping and calculation: I would be interested in your v= iew regarding how realistic are (1) the wire speed timestamping capability = and (2) adding multiple calculated deadline to packets. Both are required f= or this solution. DetNet flows can have high bandwidth, not like 1588 packe= ts. Specific comments (Section 2-3): 5, Time gated Queuing [802.1Qbv] allows multiple gates to be open simultane= ously. Furthermore preemption [802.1Qbu] can be combined with Qbv. So I can= not fully agree with your evaluation of time-aware scheduling. (Section 2, = "However, time-aware scheduling suffers from two major disadvantages.") Of = course with a bad design one may shoot in his/her own leg, but in well-desi= gned scenarios efficiency loss can be eliminated without expensive computat= ion. 6, Deadtime calculation is also complex if impact of other DetNet flows mus= t be considered. The pre-calculation of individual "local" deadlines may ne= ed to be re-calculated when flows added to the network and they are using s= ome common links. So I cannot agree with your statement. (Section 3, "... S= ince the ingress router inserts the deadline stack into the packet headers,= no other router needs to be aware of the requirements of the time sensitiv= e flows. Hence admitting a new flow only requires updating the information= base of the ingress router."). Adding a flow may result in updating many i= ngress routers' configuration. Some topics for further discussions/considerations: 7, Impact of some DetNet functions (e.g., PREOF [RFC8655]) needs further di= scussions. The usage of replication/elimination impacts the deadline calcul= ation. Disjoint paths used by member flows requires different label stack a= nd deadlines. Furthermore, these path specific labels+deadlines must be add= ed by the replication node (PRF). So at least, a combination of PRF with B-= SID and computing/adding deadlines (ingress SRTSN function) seems to be nec= essary in your solution. 8, The ever-green topic ( and not the green-wave :--))) ): label stack size= . You need multiple labels per hop and the label stack contains them for ea= ch hop. Could result in a quite big label stack to be pushed at the ingress= (nx10s of labels). I am happy to have further discussions on this interesting idea Thanks & Cheers Bala'zs From: detnet > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 2:14 PM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [Detnet] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR03MB35224845605B31C5C53198D4E5939AM0PR03MB3522eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Bala’zs,

The IEEE definition of TSN mentions determinism, but= defines it not as constant delay as in a TDM service, but rather as

to provide dete= rministic services through IEEE 802 networks,

i.e., guaranteed packet transport with bounded laten= cy, low packet delay variation, and low packet loss.

So “determinism” in this context means b= ounded latency, low PDV and low PLR.

DetNet is aligned with this definition. From the cha= rter:

deterministic data paths that operate over Layer 2 b= ridged and Layer 3 routed segments, where such paths can provide bounds on = latency, loss, and packet delay variation (jitter), and high reliability.

So it is the path that is deterministic, but the pac= ket parameters need to be bounded.

In Carrier grade Ethernet “determinism” = is usually taken to mean that a flow is pinned to a path,

that is, we know precisely which network elements a = packet will traverse,

although delay, PLR, etc. are parameters to be set a= nd/or measured.

When I say that a packet or flow is ”time sens= itive” I merely mean that there is a time budget to be attained.=

However, SRTSN by any other name smells as sweet to = me.

Stack-based Time-sensitive Enhancements for Immediat= e-delivery Networking (STEIN) could also work :) .

Y(J)S

From: Bal=E1zs Varga A <balazs.a.varga@eri= csson.com>
Sent: 08/03/2021 11:59
To: Yaakov Stein <yaakov_s@rad.com>; detnet@ietf.org; spring@i= etf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Thanks for the clarifications, that helps a lot to p= osition correctly your draft.

As You have wrote: “Determinism is not the mai= n goal of this draft” what

was not clear from the text.

I may be somewhat mis-leaded by the name You have us= ed for the concept.

TSN is widely used in IEEE as “Time-Sensitive = Networking” and its main target

is deterministic communication.

You may consider to use a different abbreviation to = avoid that confusion. E.g.,

“srtsc” (segment routing time sensitive = communication) or anything else You like.

Many thanks

Bala’zs

From: Yaakov Stein <yaakov_s@rad.com>
Sent: Monday, March 8, 2021 8:27 AM
To: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Bala'zs,

Sorry that I never got back to you on this.

It is much easier to shoot back replies to shorter e= mails :)

> 1, What is the node behavior if deadtime expire= s?

Nothing special is done if a local deadline is excee= ded. There is still hope that the time can be made up.

If too many packets fail to conform to their overall= budget,

then maybe something has changed and reoptimization = (or realizing that the application budget is not feasible) is required.

> 2, Do I understand it correctly that the descri= bed solution is practically a new per-hop-behavior?

Well, new to here but hardly “new” (simi= lar things were done in ATM…)

The new per-router behavior requires a new data stru= cture in the router instead of a FIFO queue,

but there is no absolute requirement for other tools= .

As I just hinted and have mentioned in a previous em= ail, later on we may need some OAM to see if configuration needs to be upda= ted.

I have already experimented with periodic RSVP-like = synthetic packets that collect timestamps along the path.

> 3, Design of a guaranteed upper bound usually r= equires deterministic arrival at each node along the path.

Here is where we differ (or at least put different e= mphasis on the word “usually”.

Determinism is not the main goal of this draft. Meet= ing challenging delay budgets is.

Sometimes absolute determinacy is required (e.g., in= some forms of “teleprotection” in some electricity distributio= n networks)

in which case either TDM/Qbv/calendaring can be used= , or this method coupled with a differential delay compensation buffer.

I agree that PDV increases from hop to hop to an ext= ent, but when the delay budget is really challenging you simply can’t= allow that many hops.

In mixed 5G xHaul cases in which I have been involve= d there are no more than 4 hops (usually fewer).

If you wish something more deterministic then look a= t the Andrews-Zhang strategy.

They gate at the ingress router and EDF at all the f= ollowing ones.

They show that (statistically) they ride “gree= n waves” and thus the PDV doesn’t increase significantly

(see their graphs). This comes at the expense of a l= ong cycle time for the initial gate, which means the budget can’t be = too tight.

> 4, Wire-speed timestamping and calculation=

I come from the PTP world where hardware timestampin= g with submicro resolution is common.

Since the timestamp is on the rising edge of the fir= st bit, i.e., before the classifier,

it is carried out and placed in the packet metadata = for every packet whether or not it is PTP.

I always shed tears over throwing out this informati= on.

The RSVP-like method just mentioned simply copies it= into the packet.

The SRTSN packets use this information in the schedu= ler.

I have consulted my ASIC people about the use of pri= ority heaps and insert-queues

and they ensure me that they can put something like = that into our 100G chip.

> 5, Time gated Queuing [802.1Qbv]

I didn’t say that in principle you can’t= build a Qbv system that does great things.

It is flexible enough to do anything you want.<= /o:p>

The problem is the scalability.

I have played around with optimizing it and haven= 217;t found a good “on line” algorithm,

that is, a way of adding a new flow to an existing o= ptimized design.

And even in the “off line” case (clean s= late with all flows known) slight changes and even bad clocks in the user e= quipment ruins things.

But the main idea of this draft is that even given a= good on-line algorithm for Qbv

you would need to distribute the changes in the all = the schedules to all the routers.

What happens in the meantime? What happens when one = router is already updated but another isn’t?

With the stack method only a single ingress router n= eeds to be updated and consistency is ensured!

> 6, Deadtime calculation is also complex if impa= ct of other DetNet flows must be considered.

Of course! The offset vector needs to be optimized.<= o:p>

I have several such methods and am running simulatio= ns of various strategies.

As stated in the ID the specific calculation was for= illustration purposes –

the stack mechanism is much more generic.=

> 7, Impact of some DetNet functions
I have not yet gotten to the point where I have cons= idered this.

> 8, The ever-green topic ( and not the gre= en-wave :--))) ): label stack size.

Yes, I didn’t want to bring this up yet, but h= ave been forced to.

I have to admit that I am not a fan of SRv6 for just= this reason.

But, the SRTSN stack can be really small!=

SR stack entries need only be ceil ( log2 (number of= routers) ) in size (requires a little local information base in the router= )

assuming once the stack is popped you use the origin= al DA. If you are not willing to use this additional information

you still only need sufficient suffixes, which is st= ill small.

The deadline entries can be miniscule since the wrap= around constraint is derived from the maximum flight time of a packet in th= e network.

In my designs I have gotten away with 16-32 bits per= entry, which means 4-8 hop stacks only require the room of a single IPv6 a= ddress.

Hope this answers your questions and kicks off a fru= itful discussion.

Y(J)S

From: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>
Sent: 08/03/2021 08:35
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Any view on these comments?

Thanks

Bala’zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Bal=E1zs Varga A
Sent: Friday, February 26, 2021 3:03 PM
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

many thanks for this well-written and information-ri= ch draft.

I have enjoyed to read it (and also the discussions = on the lists). :--)))

Conceptual (clarification) questions:

1, What is the node behavior if deadtime expires? Dr= op? That requires a quite good design/allocation of the latency budget. Oth= erwise it may result in unnecessary loss of a packet being relative late at= a hop, however that might be compensated by the remaining hops. (E.g., see your Figure 2, getting 35 microsecond at= R1 instead of the designed 25, but R2,R3,R4 adds only 10-10-10 extra micro= seconds). You may consider to add a "late flag" to the packet ins= tead of a drop ...

2, Do I understand it correctly that the described s= olution is practically a new per-hop-behavior? Does it have no tools to &qu= ot;influence" the arrival time at the next hops? In end-to-end latency= design the DetNet/TSN functions are often used to "adapt" the arrival of packets belonging to a DetNet flow/TSN= stream at given network points. If "latency design" requires suc= h a "time shift" then the described method has to be combined wit= h other DetNet/TSN tools.

3, Design of a guaranteed upper bound usually requir= es deterministic arrival at each node along the path. In this solution the = flow becomes less-an-less deterministic as it reaches the destination. I me= an e.g., as per Figure 2, arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-120 usec an= d at R4 is 92-167 usec. So, the flow is getting less-and-less deterministic= at each hop. That can make deterministic design of other flows passing R4 = more difficult or even impossible in some scenarios. Have I missed something here?

4, Wire-speed timestamping and calculation: I would = be interested in your view regarding how realistic are (1) the wire speed t= imestamping capability and (2) adding multiple calculated deadline to packe= ts. Both are required for this solution. DetNet flows can have high bandwidth, not like 1588 packets.

Specific comments (Section 2-3):
5, Time gated Queuing [802.1Qbv] allows multiple gat= es to be open simultaneously. Furthermore preemption [802.1Qbu] can be comb= ined with Qbv. So I cannot fully agree with your evaluation of time-aware s= cheduling. (Section 2, "However, time-aware scheduling suffers from two major disadvantages.") Of course with a b= ad design one may shoot in his/her own leg, but in well-designed scenarios = efficiency loss can be eliminated without expensive computation.=

6, Deadtime calculation is also complex if impact of= other DetNet flows must be considered. The pre-calculation of individual &= quot;local" deadlines may need to be re-calculated when flows added to= the network and they are using some common links. So I cannot agree with your statement. (Section 3, "... Since = the ingress router inserts the deadline stack into the packet headers, no o= ther router needs to be aware of the requirements of the time sensitive flo= ws. Hence admitting a new flow only requires updating the information base of the ingress router."). Addi= ng a flow may result in updating many ingress routers' configuration.<= /o:p>

Some topics for further discussions/consideration= s:

7, Impact of some DetNet functions (e.g., PREOF [RFC= 8655]) needs further discussions. The usage of replication/elimination impa= cts the deadline calculation. Disjoint paths used by member flows requires = different label stack and deadlines. Furthermore, these path specific labels+deadlines must be added by the rep= lication node (PRF). So at least, a combination of PRF with B-SID and compu= ting/adding deadlines (ingress SRTSN function) seems to be necessary in you= r solution.

8, The ever-green topic ( and not the green-wave :--= ))) ): label stack size. You need multiple labels per hop and the label sta= ck contains them for each hop. Could result in a quite big label stack to b= e pushed at the ingress (nx10s of labels).

I am happy to have further discussions on this inter= esting idea

Thanks & Cheers

Bala'zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 2:14 PM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [Detnet] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow.
Spring is also dir= ectly relevant due to the use of a stack in the header and the combined app= roach just mentioned.
PCE is relevant to the case of= a central server jointly computing an optimal path and local deadline stac= k.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_AM0PR03MB35224845605B31C5C53198D4E5939AM0PR03MB3522eurp_-- From nobody Mon Mar 8 04:54:47 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 868803A0E68; Mon, 8 Mar 2021 04:54:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.798 X-Spam-Level: X-Spam-Status: No, score=-1.798 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v_L7SbrwQGhf; Mon, 8 Mar 2021 04:54:39 -0800 (PST) Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E68193A0E66; Mon, 8 Mar 2021 04:54:38 -0800 (PST) Received: from fraeml703-chm.china.huawei.com (unknown [172.18.147.206]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DvJ8m06Btz67wpC; Mon, 8 Mar 2021 20:50:16 +0800 (CST) Received: from dggeme754-chm.china.huawei.com (10.3.19.100) by fraeml703-chm.china.huawei.com (10.206.15.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Mon, 8 Mar 2021 13:54:36 +0100 Received: from dggeme751-chm.china.huawei.com (10.3.19.97) by dggeme754-chm.china.huawei.com (10.3.19.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Mon, 8 Mar 2021 20:54:34 +0800 Received: from dggeme751-chm.china.huawei.com ([169.254.210.30]) by dggeme751-chm.china.huawei.com ([169.254.210.30]) with mapi id 15.01.2106.013; Mon, 8 Mar 2021 20:54:34 +0800 From: "Liubingyang (Bryan)" To: Yaakov Stein , Haoyu Song , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpAAA9T+5A= Date: Mon, 8 Mar 2021 12:54:34 +0000 Message-ID: <9b01579697ba42cf90b0bd5861730fc3@huawei.com> References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.108.234.163] Content-Type: multipart/alternative; boundary="_000_9b01579697ba42cf90b0bd5861730fc3huaweicom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 12:54:42 -0000 --_000_9b01579697ba42cf90b0bd5861730fc3huaweicom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Yaakov, Great work. One suggestion. If we divide the end-to-end delay budget to per-hop budgets, and carry them= in SR header, we can mimic using the end-to-end absolute deadline. One thi= ng we need to do is, when the actual time used in a hop is less than the bu= dget of this hop, you can add the remaining budget to the budget of next ho= p. In this case, you can save a lot of bits used for time, and you don't ha= ve to rely on accurate time sync. Bryan (Bingyang Liu) From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Monday, March 8, 2021 1:37 PM To: Haoyu Song ; detnet@ietf.org; spring@ietf.org= ; pce@ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We'll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song = > Sent: 08/03/2021 04:41 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1) The use of clock time as deadline requires network synchronization,= and accurate measurement of per-link propagation time, which can somehow l= imit the application scope of this work. Alternatively, one can simply budg= et a device latency which require a router/switch to obey. In case the over= all budget is evenly divided by the hops, a single parameter is enough. Of = course, if one wants to customize the budget on each hop (which might be ne= cessary considering the different capability/capacity of each hop), a stack= is still needed. 2) Mechanism should be provisioned to track where the timing requireme= nt is violated and by how much (e.g., using timestamp or flag). This would = be very useful for troubleshooting. 3) Recently programmable scheduler research has proposed several primi= tives such as PIPO and PIEO and provided feasible hardware implementations.= The scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_9b01579697ba42cf90b0bd5861730fc3huaweicom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Yaakov,

Great work. One suggestion.

If we divide the end-to-end delay budget to per-hop budgets, and = carry them in SR header, we can mimic using the end-to-end absolute deadlin= e. One thing we need to do is, when the actual time used in a hop is less than the budget of this hop, you can add= the remaining budget to the budget of next hop. In this case, you can save= a lot of bits used for time, and you don’t have to rely on accurate = time sync.

Bryan (Bingyang Liu)

From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein
Sent: Monday, March 8, 2021 1:37 PM
To: Haoyu Song <haoyu.song@futurewei.com>; detnet@ietf.org; sp= ring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Haoyu

I think we are in agreement.

I do not see the need for expli= citly handling the case of a packet missing a LOCAL deadline,

since the following switches wi= ll already handle this case optimally (and may still meet the overall budge= t).

Counting packets that miss thei= r delay budget is indeed important,

and a counter could be configur= ed in the egress router for this.

We’ll need to define this= when we get to the protocol specification.

It would be advantageous to put= a threshold on the failure rate

and feed this back to the path/= stack optimizer.

Y(J)S

From: Haoyu Song <haoy= u.song@futurewei.com>
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Some feedback inline.

Best regards,=

Haoyu

From: Yaakov Stein <yaakov_s@r= ad.com>
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Haoyu,

I’ll address your points:=

> The use of clock time as d= eadline requires network synchronization

That is a basic assumption of T= SN networks (IMO the defining assumption).

However, the sync needn’t= be highly accurate – it depends on the tightness of the delay budget= s.

>> Yes, I understand it. = What I mean is that the method mentioned in the draft seems to be also appl= icable to other types of networks. For example, we can envision some time-c= ritical traffic in DCN or WAN. If a protocol would be developed, can it also serve the other networks? If so, it would = be better.

> accurate measurement of pe= r-link propagation time

Yes, I am assuming that once an= for all someone does a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measur= ement of the links, and these are stored in some database.

Once again, the required accura= cy depends on the delay budgets.

> which can somehow limit th= e application scope of this work

If the delay budget only above = the physically minimal delay by say less than 100 microseconds,

I agree that the previous two i= ssues MUST be carried out. But in such cases there is no alternative.<= /o:p>

If the delay budget is much hig= her than that, then one could use an RSVP-like mechanism,=

sending a packet (or several pa= ckets) from source to destination collecting a stack of timestamps,

and then using that stack for t= he following packets.

> Mechanism should be provis= ioned to track where the timing requirement is violated and by how much

I’ll leave the OAM for la= ter. However there are already many high accuracy performance measurement t= echniques and protocols for this.

>> For this I mean someth= ing recorded in the same packet with the deadlines. If it misses the deadli= ne, the receiver may need to know where it’s violated. Other independ= ent methods are possible, but it’s better to consider if it can be integrated in the current proposal.

> Recently programmable sche= duler research has proposed several primitives

Yes, I tried to stress that thi= s ID is not limited to EDF (although sometimes that is a good strategy).

One can even reproduce Qbv beha= vior using a stack of deadlines (although why would one wish to do so?).

> such as PIPO and PIEO=

I’ve heard of PIFO (Push = In First out) but not PIPO. Is this a typo or something new?

I agree that there are mechanis= ms that are optimized for hardware, but I have come up with a very nice har= dware implementation for PEDF

and prefer to find hardware imp= lementations for optimal schedulers, rather than to determine schedulers ba= sed on optimal hardware.

>> Sorry that’s a t= ypo. I mean PIFO (although we do have a paper under review using the name P= IPO). Yes I agree those are just abstract primitives. The actual implementa= tion, if customized to a particular algorithm, would be simpler.

Y(J)S

From: Haoyu Song <haoy= u.song@futurewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not click links or open attachments unless you know the content is safe.

Hi Yaakov= ,

Just got a chance to read your = draft. I agree with the comments of the others that this is a very interest= ing work. I’ll just add a few points.

1= )   &= nbsp; The use of clock time a= s deadline requires network synchronization, and accurate measurement of pe= r-link propagation time, which can somehow limit the application scope of t= his work. Alternatively, one can simply budget a device latency which require a router/switch to obey. In case the= overall budget is evenly divided by the hops, a single parameter is enough= . Of course, if one wants to customize the budget on each hop (which might = be necessary considering the different capability/capacity of each hop), a stack is still needed.

2= )   &= nbsp; Mechanism should be pro= visioned to track where the timing requirement is violated and by how much = (e.g., using timestamp or flag). This would be very useful for troubleshoot= ing.

3= )   &= nbsp; Recently programmable s= cheduler research has proposed several primitives such as PIPO and PIEO and= provided feasible hardware implementations. The scheme proposed in this dr= aft can easily fit into these primitives.

Best regards,=

Haoyu

From: spring <spring-bo= unces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your atten= tion to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-b= ased approach (similar to segment routing) to time sensitive networking.

It furthermore proposes combini= ng segment routing with this approach to TSN

resulting in a unified approach= to forwarding and scheduling.

The draft is information at thi= s point, since it discusses the concepts and does not yet pin down the prec= ise formats.

Apologies for simultaneously se= nding to 3 lists,

but I am not sure which WG is t= he most appropriate for discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-end = latency of a time-sensitive flow.
Spring is also = directly relevant due to the use of a stack in the header and the combined = approach just mentioned.
PCE is relevant to the c= ase of a central server jointly computing an optimal path and local deadlin= e stack.

I’ll let the chairs decid= e where discussions should be held.

Y(J)S

--_000_9b01579697ba42cf90b0bd5861730fc3huaweicom_-- From nobody Mon Mar 8 05:38:11 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 72A1C3A2A3F; Mon, 8 Mar 2021 05:38:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.799 X-Spam-Level: X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8OGEL5NFGDgR; Mon, 8 Mar 2021 05:37:59 -0800 (PST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80045.outbound.protection.outlook.com [40.107.8.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 228C53A2A3C; Mon, 8 Mar 2021 05:37:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jVLfMH96u9v3hwpurSScEzuNEjMDQVLgH07Uj50k9ct1Yk1/+2HhLQAfAkHEUjauLqrAQDvj2ZoU4ic3UULo1btB9M+uJORKT/DLM3975E96Qn8/m14wzm2hJSG5/t+JSRxJFbEfCRtRuy5bIgZ1AefABJFKUo3fomf/mp7SX+4MPRHvd6XxMrqt/N2oBNJ8ie5KT0QrOixsrYBgQxBQF2onr1YXG9Ue/gzsJVpKh+j6NLuZOFQKda3HISYy7o2ZVzlr1zGJmvh1lyCnqCjDWrgDXZ3C75keCmbphtzbzyFhzpwlkklzs4vn8uNn9hsiz+IrDkN4PSYGrrqvKWbkGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qEWjRGV5Af8vxCDrTSrsLcdj/tcyPlRtEF5S3ZKWu4Q=; b=hG4CfO6V1YvW8LrbGeEDe9+IspS0Fc9bDxZ6QwdKcHweJbuRS8f8d2cTLyBMTYrBP6NoPFkwnza6GNWppvLUpj5KRWoFQiB/9KURdRPvLOzI47QIt2R8dEttOvdP286KnjhocHNcu8fy1yn+HEouu+IGDOiGntmbHfdoh+yZruwLUWLP+7gyoLMl/niYFmMIvbGD3V0YFVVM+W4/OQmvOd2Od3trDKrBQErHVIf8/g35HkOn4miDomUhJdZGBJIanPz6ahFZ2ZRY8kDmjla7W4J25UA2o9Z/a6ejpquF+D1g0kL5sbyK6DrvvvvSHkUelcKQ3bM06gmfOcjHNTm3jQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qEWjRGV5Af8vxCDrTSrsLcdj/tcyPlRtEF5S3ZKWu4Q=; b=beMccMOXlSPVCNhVz7VxfiGOVEx1+EVA6es/LtOsGtrO4QwFi2MvOHzlg38JKbAM99AYF7rrgu/0tyA1ZYUcIkZaBC3Db9pdpD1s4nPkkb/LnXDxJEGW5RHcLXjVkmyFtFr6yAiv6LnNMTT/XgdoyJ/4ces+TIv/FEVp0cd+Sp4= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM9PR03MB7377.eurprd03.prod.outlook.com (2603:10a6:20b:26a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17; Mon, 8 Mar 2021 13:37:56 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 13:37:56 +0000 From: Yaakov Stein To: "Liubingyang (Bryan)" , Haoyu Song , "detnet@ietf.org" , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpAAA9T+5AAAZ19cA== Date: Mon, 8 Mar 2021 13:37:55 +0000 Message-ID: References: <9b01579697ba42cf90b0bd5861730fc3@huawei.com> In-Reply-To: <9b01579697ba42cf90b0bd5861730fc3@huawei.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d6101d1b-87d7-47b3-fdb9-08d8e237618b x-ms-traffictypediagnostic: AM9PR03MB7377: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: VDp0eBeAn/tPQDONYe+JiZsoM0Z73RnUBPFF/G/xcQbl/A/abfBnKXnZ7bcTXJxdlsV/R3MLSutr7hau/YV2mws5V7cEjaGZJ5gVG0QjFzQXiNKslQ6d46GFi6kdOWrrKilVmyIXRkJlC5KfLZnhsGjhZ6+rogjG1juGxPf3v5R3b1SXjHYs8NBidQZm/PmvKshTmy0JrAIUGo6AlALbv5N3uYrT4Uu0nkicmtgvFNn6ao4B+SFdG+6vr4fEHJVJ18QqggwkcvJtJLXnm0t1izNevvb06G66rcu7yuYuHt1OKf/QGECng/CN5Gk9VLYxJAC8a/4/1J6DfCxHqJDXt5HNcbAPj0bi4d4Ug1uVZWFFtHMrZAcJyDCM9+SMS+WgBajR/EsA4x1Q/IARno56IYLx4g8rmD34mqSPfFUktAuVMFE6dxOPzmCsWZs3833VBX0Eq40jn47KQGidbCJVstkVu2EBpJgfsn707+sW7njGD/8wElMkS8T22Pc+8jH6pwtgv4H6huSRSg5SFhOR6Yfwb4o5ghkV9rre1bgvPIduFsS4A68hCbHgRbjrnppGN4YH6R3BnzP5xQp+1RQIALupWo3ibFCy226lY29tF+4= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(346002)(376002)(366004)(39850400004)(136003)(6506007)(86362001)(83380400001)(478600001)(53546011)(8676002)(966005)(71200400001)(186003)(8936002)(26005)(76116006)(52536014)(66476007)(66446008)(66556008)(66946007)(64756008)(2906002)(110136005)(7696005)(316002)(5660300002)(9686003)(33656002)(66574015)(166002)(55016002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?J5UZ/C2NuFYvBSHe8v8odIdiNnSsx0qT4ff1e8ZonjZHMtHKBpRGk+9fPD8Z?= =?us-ascii?Q?/JcMLm499SJ+Ulh1aRpVwhUZd//VpYpeSnXcfNOI+IA9F83bdgYJs3EFdHD3?= =?us-ascii?Q?YwZNkYkkNQsWxMRYQ0o/k0jEkdqGj4hFXr7/zsZp6OWJWm5Rxra6ROLlrKKL?= =?us-ascii?Q?ZSCPKSIgEYJZRCsStNJclB+0VVsrpr9e9k7kKuq16vYsGLeRXSAZrPZhPwet?= =?us-ascii?Q?DbTLwNEfH9AsE7DRT7922mgKiIdPox3evnxevB1WnCj87AgKYOzgSfFbF+7F?= =?us-ascii?Q?Yn+wC0RgUGxt0yZ0D6f6h5exxDbRv4XuQWPBz1PLPNKtc5r//JxYhoVI+OFA?= =?us-ascii?Q?lWVY3cx3qCrY8bDChNL7C0OrA6CkYHjag4YN3bJBdzdGru/dWiu8BEIX6k88?= =?us-ascii?Q?5W+JL6hB/JQvFvmASj7UgpB7K5v13kJHW6NfroX5YsjpAKMII2unBqCaKlJE?= =?us-ascii?Q?nS/yuZ5NsQVPt22s7FNrV8tcwsMucji0/VloqIthRN/xyCa+ph/r9BerGUFu?= =?us-ascii?Q?JLqGg63SYYDeW/UU1fEDF5s2lnPLC3oMUEhX7OesX1jtqsCyGRvsp/hLhiF7?= =?us-ascii?Q?mjlDSLExIxQ7yxcYos/8Q/NQv4dXRmhGn10bVGT6//RhOlW688upUSP6MjM8?= =?us-ascii?Q?e62uBOwiwtiKdB4gSpntQAAFsILPBG3lpoKHWWXmLVJxgV2gJmlfgcAlYLK9?= =?us-ascii?Q?l3n5dxwsLtRn/PNDA7y+c7C9px42c2oaWYgfnNmVF1N83JOeT3cFJ68mrKLw?= =?us-ascii?Q?m3/9yatRHLUM7/wJuT6QndcaNDooMoIiM2tKj/3w8876Pjl9NhQc+s0WJv0A?= =?us-ascii?Q?//obrDOqXyMZInq8js4PANrHXl9eVQOfGTc7G6NUU/sdVhClCqMso8OBCLFX?= =?us-ascii?Q?yCLMyp4SiEOCbf1JJ8XYQUh7kfpBQK/dWkYUjLAUFo8Oqg3NUitxGmgWeWOd?= =?us-ascii?Q?8UvZ9sGQBPAh3qb/D0JaCoPHWD7ggGeJOHnFTbSwuC08H6Q4N0Sz0lNjscQ9?= =?us-ascii?Q?EvGPhPj3+4cOuGhY8dGM4TgekrlsYG0diU+qKKEMS+QOkQfzV3lytQf2FpYj?= =?us-ascii?Q?zwJGi5zXM4F5EddmDC0ZY6DDZ9wC/eeHfvnUrYWkA2Z8cDRxw3KNJA/o7ZqI?= =?us-ascii?Q?I60tfeJfnY/rJkOkl24xpqhCIkMVEwaDTbsZZJbnRjktypynBlw7o11fFRpq?= =?us-ascii?Q?9c5lrkOGVvBFnaPPIcbS5h6ccdO++7qdWLbA7V5DgHpKEIb3KBI1CCgF5MCO?= =?us-ascii?Q?wZycfWyVqTi7ghrsX232CsNCHCCURxB1rMre3UFsJeBouwPgzj4Z7HzfpwCO?= =?us-ascii?Q?mqXToGd/01O42BMq9Ui151Oj?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR03MB35220E7B943E9BD4EB64624AE5939AM0PR03MB3522eurp_" MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d6101d1b-87d7-47b3-fdb9-08d8e237618b X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 13:37:55.9798 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: uscjpBaKnl2AbMVko6LmA7qfbQj+lndUpYsF7Ry4fvBpCJ65sBOHjfvHAbmhj5sZunpS76QDyjVDYFgJKG/+7A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7377 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 13:38:03 -0000 --_000_AM0PR03MB35220E7B943E9BD4EB64624AE5939AM0PR03MB3522eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Bryan The local deadlines are specified in absolute time - not deltas. So there is no need to add the time saved - it is already there! As I previously replied, the accuracy of the time sync depends on the tight= ness of the budget. It has nothing to do with the encoding. Y(J)S From: Liubingyang (Bryan) Sent: 08/03/2021 14:55 To: Yaakov Stein ; Haoyu Song ;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Great work. One suggestion. If we divide the end-to-end delay budget to per-hop budgets, and carry them= in SR header, we can mimic using the end-to-end absolute deadline. One thi= ng we need to do is, when the actual time used in a hop is less than the bu= dget of this hop, you can add the remaining budget to the budget of next ho= p. In this case, you can save a lot of bits used for time, and you don't ha= ve to rely on accurate time sync. Bryan (Bingyang Liu) From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Monday, March 8, 2021 1:37 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We'll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song = > Sent: 08/03/2021 04:41 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_AM0PR03MB35220E7B943E9BD4EB64624AE5939AM0PR03MB3522eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Bryan

The local deadlines are specified in absolute time &#= 8211; not deltas.

So there is no need to add the time saved – it = is already there!

As I previously replied, the accuracy of the time syn= c depends on the tightness of the budget.

It has nothing to do with the encoding.

Y(J)S

From: Liubingyang (Bryan) <liubingyang@hua= wei.com>
Sent: 08/03/2021 14:55
To: Yaakov Stein <yaakov_s@rad.com>; Haoyu Song <haoyu.song= @futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Great work. One suggestion.

If we divide the end-to-end delay budget to per-hop b= udgets, and carry them in SR header, we can mimic using the end-to-end abso= lute deadline. One thing we need to do is, when the actual time used in a hop is less than the budget of this = hop, you can add the remaining budget to the budget of next hop. In this ca= se, you can save a lot of bits used for time, and you don’t have to r= ely on accurate time sync.

Bryan (Bingyang Liu)

From:<= /span> detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein
Sent: Monday, March 8, 2021 1:37 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

&nbs= p;

Haoyu

&nbs= p;

I think w= e are in agreement.

&nbs= p;

I do not = see the need for explicitly handling the case of a packet missing a LOCAL d= eadline,

since the= following switches will already handle this case optimally (and may still = meet the overall budget).

&nbs= p;

Counting = packets that miss their delay budget is indeed important,

and a cou= nter could be configured in the egress router for this.
We’= ll need to define this when we get to the protocol specification.

&nbs= p;

It would = be advantageous to put a threshold on the failure rate
and feed = this back to the path/stack optimizer.

&nbs= p;

Y(J)S

&nbs= p;

From:<= /span> Haoyu Song <haoyu.song@futurewei.com>
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

&nbs= p;

Hi Yaakov= ,

&nbs= p;

Some feed= back inline.

&nbs= p;

Best rega= rds,

Haoyu

&nbs= p;

From:<= /span> Yaakov Stein <yaakov_s@rad.com>
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

&nbs= p;

Haoyu,

&nbs= p;

I’l= l address your points:

&nbs= p;

> The = use of clock time as deadline requires network synchronization

That is a= basic assumption of TSN networks (IMO the defining assumption).=

However, = the sync needn’t be highly accurate – it depends on the tightne= ss of the delay budgets.

>> = Yes, I understand it. What I mean is that the method mentioned in the draft= seems to be also applicable to other types of networks. For example, we ca= n envision some time-critical traffic in DCN or WAN. If a protocol would be developed, can it also serve the other netw= orks? If so, it would be better.

&nbs= p;

&nbs= p;

> accu= rate measurement of per-link propagation time

Yes, I am= assuming that once an for all someone does a TDR or at least OWAMP/TWAMP/Y= .1731-1wD delay measurement of the links, and these are stored in some data= base.

Once agai= n, the required accuracy depends on the delay budgets.
&nbs= p;

> whic= h can somehow limit the application scope of this work
If the de= lay budget only above the physically minimal delay by say less than 100 mic= roseconds,

I agree t= hat the previous two issues MUST be carried out. But in such cases there is= no alternative.

If the de= lay budget is much higher than that, then one could use an RSVP-like mechan= ism,

sending a= packet (or several packets) from source to destination collecting a stack = of timestamps,

and then = using that stack for the following packets.

&nbs= p;

> Mech= anism should be provisioned to track where the timing requirement is violat= ed and by how much

I’l= l leave the OAM for later. However there are already many high accuracy per= formance measurement techniques and protocols for this.
>> = For this I mean something recorded in the same packet with the deadlines. I= f it misses the deadline, the receiver may need to know where it’s vi= olated. Other independent methods are possible, but it’s better to consider if it can be integrated in the current p= roposal.

&nbs= p;

> Rece= ntly programmable scheduler research has proposed several primitives

Yes, I tr= ied to stress that this ID is not limited to EDF (although sometimes that i= s a good strategy).

One can e= ven reproduce Qbv behavior using a stack of deadlines (although why would o= ne wish to do so?).

&nbs= p;

> such= as PIPO and PIEO

I’v= e heard of PIFO (Push In First out) but not PIPO. Is this a typo or somethi= ng new?

I agree t= hat there are mechanisms that are optimized for hardware, but I have come u= p with a very nice hardware implementation for PEDF

and prefe= r to find hardware implementations for optimal schedulers, rather than to d= etermine schedulers based on optimal hardware.

>> = Sorry that’s a typo. I mean PIFO (although we do have a paper under r= eview using the name PIPO). Yes I agree those are just abstract primitives.= The actual implementation, if customized to a particular algorithm, would be simpler.

&nbs= p;

Y(J)S

&nbs= p;

From:<= /span> Haoyu Song <haoyu.song@futurewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

&nbs= p;

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,<= /span>

&nbs= p;

Just got = a chance to read your draft. I agree with the comments of the others that t= his is a very interesting work. I’ll just add a few points.

&nbs= p;

The use of clock time as deadline requires networ= k synchronization, and accurate measurement of per-link propagation time, w= hich can somehow limit the application scope of this work. Alternatively, one can simply budget a device latency = which require a router/switch to obey. In case the overall budget is evenly= divided by the hops, a single parameter is enough. Of course, if one wants= to customize the budget on each hop (which might be necessary considering the different capability/capacit= y of each hop), a stack is still needed.
Mechanism should be provis= ioned to track where the timing requirement is violated and by how much (e.= g., using timestamp or flag). This would be very useful for troubleshooting= .
Recently programmable sch= eduler research has proposed several primitives such as PIPO and PIEO and p= rovided feasible hardware implementations. The scheme proposed in this draft can easily fit into these primitives.
=

&nbs= p;

Best rega= rds,

Haoyu

From:<= /span> spring <spring-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN<= /o:p>

&nbs= p;

All,

&nbs= p;

I would l= ike to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which des= cribes using a stack-based approach (similar to segment routing) to time se= nsitive networking.

It furthe= rmore proposes combining segment routing with this approach to TSN

resulting= in a unified approach to forwarding and scheduling.

&nbs= p;

The draft= is information at this point, since it discusses the concepts and does not= yet pin down the precise formats.

&nbs= p;

Apologies= for simultaneously sending to 3 lists,

but I am = not sure which WG is the most appropriate for discussions of this topic.

DetNet is most relevant since the whole point is = to control end-to-end latency of a time-sensitive flow.
Spring is also directly relevant due to the us= e of a stack in the header and the combined approach just mentioned.
<= span style=3D"mso-fareast-language:ZH-CN">PCE is relevant to the case of a = central server jointly computing an optimal path and local deadline stack.<= o:p>

I’l= l let the chairs decide where discussions should be held.=

&nbs= p;

Y(J)S

&nbs= p;

--_000_AM0PR03MB35220E7B943E9BD4EB64624AE5939AM0PR03MB3522eurp_-- From nobody Mon Mar 8 06:13:49 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDA633A2AD9 for ; Mon, 8 Mar 2021 06:13:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=labn.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2Os1S9daiQQH for ; Mon, 8 Mar 2021 06:13:47 -0800 (PST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2137.outbound.protection.outlook.com [40.107.93.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BFF043A2AE6 for ; Mon, 8 Mar 2021 06:13:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dPFS1KJX27VQp5HTI9GCkUPgaB+77wZFMXLKS/aP/+GWzbJyoAH0m3XQZbm0YW4jOTv3MMzmxr4e/14R0aLXgB5mX39gUEP1tvI6vrQLl8BC8CGPwd/SeFCtSF+D/iz9DDlVnfIG3NeqTSHdSnbPaR14XSLbl8ZBGo7kr5SYQfa20U+0+E0kc38ptYL3GoXOmvK043c9P2RGS0AkytsKWOh3jt0VN6o6dGcLEzo0xMpmXPSwJe1+buMOPgcqCswvoAipPj8HwlZpWP3AP/bmt/NjcrEzhiE25L06AI2RGw2ZOIt/yt9TyHHhQ/Bqpgbs290mMfodVahylB8jTBOCdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WkoxCDsjTEL+rOdkxty13mqXv66VIqcKCkXPO05rr1E=; b=kIDx+FRicztmUozyVlIEJ74IckIkVpgju70wumiDv/Yp5JCblwpNpoyJ/8ANW7FkWZkj1fFpZOgvYwENtWlSa6qoyCI4RICl6YIH0HYKAP5E8S7pk8hAzD260aFIt/2SZqvxO9FpMiiZhJezt+xQFLkNaOwhlDgC6gKs/ObyG+r95Fq+iAfnXYbPeTKmiUdtx+mlXnqeGTRMKJ2vuBBt4vcsSoIxzcVP94Buxa56byIsOL5B0J6Y8WdiXjenNOEwnwsXh2gX91CdOI5i/DsjMU/c92Ifx6bDHg9+xwhhe7pnJRQYn19QncolJyka40kEQROXbIkkxByZFyXcsTH1cA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=labn.net; dmarc=pass action=none header.from=labn.net; dkim=pass header.d=labn.net; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=labn.onmicrosoft.com; s=selector2-labn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WkoxCDsjTEL+rOdkxty13mqXv66VIqcKCkXPO05rr1E=; b=ru7yhUir7Yp7+zbD6sJJF0MJCOHQ5BWVSE2exul4kc/8VbsVMf8JTOLR4X6QUqNeEybIfU/+Z9gEb5SvXfg3lQrP6sqGghSlMJmtr5RDGxqG/r2I482q+73T9iirUdw+9l7pSivNXrCDsb/Gt8BOX1XR2qCE2ASl5U8fip7sLBU= Received: from MN2PR14MB4030.namprd14.prod.outlook.com (2603:10b6:208:1dc::14) by MN2PR14MB3439.namprd14.prod.outlook.com (2603:10b6:208:1af::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Mon, 8 Mar 2021 14:13:44 +0000 Received: from MN2PR14MB4030.namprd14.prod.outlook.com ([fe80::4e7:7c3b:27f7:380a]) by MN2PR14MB4030.namprd14.prod.outlook.com ([fe80::4e7:7c3b:27f7:380a%7]) with mapi id 15.20.3890.038; Mon, 8 Mar 2021 14:13:44 +0000 From: Don Fedyk To: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= , 'DetNet WG' Thread-Topic: New version of DetNet YANG Thread-Index: AdcG9o3EjjvV1BVlRUyxys/kk3N9nAIdllRgADIF9DAAXV2rIAASxZ2A Date: Mon, 8 Mar 2021 14:13:44 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ericsson.com; dkim=none (message not signed) header.d=none;ericsson.com; dmarc=none action=none header.from=labn.net; x-originating-ip: [173.48.105.206] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c3a52ccd-6529-4680-6595-08d8e23c61f9 x-ms-traffictypediagnostic: MN2PR14MB3439: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: nVqm2L5dd++rk/YUCHRE9rpnZzl1BpxdW2c665wBnb6rVNHytJBJ15AwvBy0Gn2GnVYp4e272I+hKFjj1xEi3fTrJEkdyztWb1mLCmHNEzCJJzrYc/XQk5bM1v0UUPWulx4rbozdazdsooFN7ZxqmagHoEcHNAJmqxaYk76CBRCiOS0MGw0K1j6OBENsbymtmtlTgpEnYoRvPejvS11e1QFJF8FAvI8K7Wj39+zM+zZQK527/US6UlrnivODrwjOrMTtrgon173nQr8Hyu1A24H+1VaGzXWDy+d9EeIdWQfLS2YWajUUqwp724mutAs7lRgQ1PrIggj7CNQVXBZHlOFS6iBkdLwNScwx+JVM23aoJ2gV5h5rEdtiCmPcXKgvVOtGgHVOcLlQIVHvZ8RxUlmmthGUPIZEfI01QPGtvTxsg3+5dYIqJUDy7765pCwKoAkDnOIN/8kx925QPWKWV80D2ul7Ktc6dkEYmQKxSfMHD3OS0qdacyLIQwL6uIwIsfNy7K4appydNmZHk8sTxi9eXF6on1zMxlIPfzG1pif5h0xAhovXLGUeUidD2Ji9bvN2FjzckIqlicnBij7N1//NsYODzd3PPvYguzypzxM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR14MB4030.namprd14.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(376002)(366004)(136003)(39830400003)(346002)(26005)(7696005)(966005)(6506007)(166002)(8676002)(478600001)(55016002)(9686003)(8936002)(86362001)(33656002)(316002)(66574015)(66476007)(76116006)(110136005)(64756008)(66556008)(52536014)(66446008)(53546011)(66946007)(83380400001)(71200400001)(186003)(5660300002)(2906002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?ZnMUSrq6o/qn0zx2M4B/l5x6oUWST8CGpKtLi0+/yId/47/HFRfh9YTFUF?= =?iso-8859-1?Q?E/ctkBdTRuDaOJyhnSRYvevtF9zQVRyLQrudbs+E5UOZhemerw3l2CK1SU?= =?iso-8859-1?Q?uqrnrabgPyTxQ1pxvuUj84Z4P1ts5UKBzQaZq/X/c2OE+GJQ2v1jMnLe3g?= =?iso-8859-1?Q?gF6WcTy34FVxZOHqaKhB6G2G2Ky/0x0Wkn2MlMVO6a8kzi9ipGTf3dhv5C?= =?iso-8859-1?Q?osEDeb8C8SaYXheYNgZCLJQCdKa+8voN9fv2hWdahp+prm8pjwL46G7Vo1?= =?iso-8859-1?Q?fdLPhaXwWdOHgpFV/US8Qqo4u+U62lgeDSU1k3BnqtYnmTXTP2Yizwdxwk?= =?iso-8859-1?Q?ef4ZHIWYZP4XLp7n3JIclcVskD2WXq9ATqJu/VwvobSAqQXyZgGME78lbn?= =?iso-8859-1?Q?3q9DiG7fJ4rXt5Y0JhLmHYvbbW7zWhXFviRjLcicxYeM358tJLxfIffOjk?= =?iso-8859-1?Q?zH/AFLfKP2DOCwN+krZAFsWTQFGYTyvikS1ScquusQZpgFxD2Ao15w3jiK?= =?iso-8859-1?Q?bdvNe0NXAvYQl4FN5UKL68F+9uMQ87eq1R4gVHJdxKUkh3mDAIuOdI6gpy?= =?iso-8859-1?Q?+g6oWPLGWfKFqGweYmVN8ta/YHWUROIBKxkL5Kubhlxp77tSaQLhmH/rQk?= =?iso-8859-1?Q?SLyPASSdG3zXuSYivUPhxzbHh3p/TqXO/sMFi8yLI5qWLH3lsk6Tc59/kX?= =?iso-8859-1?Q?Ct35Ci8QkN/VOnKOWivNCf8BKu5XOkLSoVPysWnelPvXSjdFp1nd7yzZx+?= =?iso-8859-1?Q?AetzRh8QZMcBn41uOafNJyodzuIbsMvACfen4WyfjMC1qXFWaDQXNOP466?= =?iso-8859-1?Q?n5Flf3wOyLprq2zsFzREKPCfNVz3MWeBF05Jfd+2I++VSt9y91b7iA0x79?= =?iso-8859-1?Q?/5kKYXTkxcEJAU59mYC8cYiVgnCpyzIAzhGYJqlU1tGU+jeY7XTO/Q0XyL?= =?iso-8859-1?Q?p9pla7CLg7dBltw0sRZ8MpJJWvPhCG/c/49H85lkBu+chaUZf8/fxpxQls?= =?iso-8859-1?Q?w5G97hf6iH+XCcValhxOUotCwm60srSqnRDfoxM41rqVZ4RCRTBicf9AYx?= =?iso-8859-1?Q?2ent15ko38oiKVOwHNdM64wRRsjiCGK1qZD6uHaPUK+q5Iy3LQKIygv12w?= =?iso-8859-1?Q?T/7iMinww9cFxjqlxaGkEYPKQvzjlQ6UeYLj3Bmyzi/sAdvrroW+//YpC+?= =?iso-8859-1?Q?CCCbsV20hkuvom6J3AXj5jRO4i5U4GGp5ltvUDcmtIPODTfLQU37uGrTmw?= =?iso-8859-1?Q?jpUASFgRWjnrUMaqWPjghiG26o2hzj4T7aotXqcks6sdSe7WJIRAJu9ohd?= =?iso-8859-1?Q?BtcP2LvJJvPnaKmCBlXNeEhaRWXoQ1qUSLmq5OIqJts29dJhCS57DP0poZ?= =?iso-8859-1?Q?SaOOrgkozK?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_MN2PR14MB40300A79231A97657D1F0A11BB939MN2PR14MB4030namp_" MIME-Version: 1.0 X-OriginatorOrg: labn.net X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR14MB4030.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3a52ccd-6529-4680-6595-08d8e23c61f9 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2021 14:13:44.0921 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: eb60ac54-2184-4344-9b60-40c8b2b72561 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: MXhMgjT2jUUAr9MKdvAr/SfQza67+bnrOLloNA2aR1lrMrtmre9WMInDoQSKBaNyLsCIVw05aH6h7NjkzZMRog== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR14MB3439 Archived-At: Subject: Re: [Detnet] New version of DetNet YANG X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 14:13:49 -0000 --_000_MN2PR14MB40300A79231A97657D1F0A11BB939MN2PR14MB4030namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Bal=E1zs Inline [Don] Cheers Don From: Bal=E1zs Varga A Subject: RE: New version of DetNet YANG Hi Don, 1, OK, thanks. 2, On sub-networks my question is related to the mapping of DetNet flows to= sub-networks' flows/streams. 2.a, Scenarios in the "draft-ietf-detnet-ip-over-mpls" and "draft-ietf-detn= et-mpls-over-udp-ip". I see "mpls-over-ip" in the flow type next hop choices, but nothing regarding "ip-over-mpls". Ha= ve I missed something? [Don] I'm finding it hard to parse your meaning - Sorry. If you are referr= ing to the YANG yes we have ip-over-mpls This is well specified. We do not have MPLS over IP but we have aggregatio= n of Service Sub-layer and Forwarding Sublayer over A service sublayer which could be IP. We do not explicitly support IP-IN-I= P tunnel formats (additional field). We need WG guidance if this is to be done in this YANG or other. We made a= call to have a clear aggregation model and these needed to be implemented first. 2.b, Further, I have had also the TSN drafts in mind "draft-ietf-detnet-ip-= over-tsn", "draft-ietf-detnet-mpls-over-tsn" and "draft-ietf-detnet-tsn-vpn-over-mpls". YANG for TSN is defined in IEEE,= however for mapping DetNet flows to/from TSN Streams may require e.g., some common identifier. What do You t= hink? [Don] We allow Ethernet to be included from an application as an interface.= This is a transparent data service. We can prune that stream on Ethernet parameters only egress we simply hand = the decapsulated stream back to the application. However if you create an application interface for a second stream you can = map that to a different service sub layer and Use the service sub-layer to differentiate the application at egress. Both of these issues need additional discussion if the WG if we are to add = the function. The configuration and aggregation model provides some capabilities because= they are building blocks. Cheers Bala'zs From: Don Fedyk > Sent: Wednesday, March 3, 2021 3:10 PM To: Bal=E1zs Varga A >; 'DetNet WG' > Subject: RE: New version of DetNet YANG Hi Bal=E1zs 1. We have the traffic flow requirements and flow-spec requests that can= be applied at the application, service sub-layer and forwarding sub-layer.= Each forwaring type, IP and MPLS have the data plane QoS objects. ( DSC= P for IP and traffic-class for MPLS). An SDN controller or a management p= lane could correlate QoS reservation of resources to a flow with the approp= riate markings. And you could stich a path on various nodes with MPLS lab= el. But we do not have a control plane function to correlate QoS traffic in= the YANG models that we have to any database. (i.e. a traffic engineering = database) I suspect there are existing YANG models that do some of that b= ut we have not looked into that because we do not have DetNet documents in = this area. 2. I'm not sure what you mean by sub-network specific YANG parameters. = Is there a DetNet document that you are referring to? Cheers Don From: Bal=E1zs Varga A > Sent: Tuesday, March 2, 2021 9:18 AM To: Don Fedyk >; 'DetNet WG' > Subject: RE: New version of DetNet YANG Hi Don, Thanks, cool. Two question for clarification (we may discuss on Monday as well): 1, Are there YANG parameters defined for the QoS functions used in the DetN= et forwarding sub-layer? I have found e.g., the forwarding sub-layer related M= PLS label operations, but not the QoS related ones. 2, Are there sub-networks specific YANG parameters defined/referenced as we= ll? Many thanks Bala'zs From: detnet > On B= ehalf Of Don Fedyk Sent: Friday, February 19, 2021 8:37 PM To: 'DetNet WG' > Subject: [Detnet] New version of DetNet YANG Hi We have posted a new version of the DetNet YANG https://www.ietf.org/archive/id/draft-ietf-detnet-yang-11.txt (This issue corrects some yang validation checks that a few days earlier su= bmission missed. ) This version the authors think is at a level of completeness we will be ask= ing for WG last call. Please have a look and any post comments. We plan to present the highlight= s at the next DetNet meeting. Cheers, Don --_000_MN2PR14MB40300A79231A97657D1F0A11BB939MN2PR14MB4030namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Bal=E1zs

Inline [Don]

Cheers

Don

From: Bal=E1zs Varga A <balazs.a.varga@eri= csson.com>
Subject: RE: New version of DetNet YANG

Hi Don,

1, OK, thanks.

2, On sub-networks my question is related to the map= ping of DetNet flows to sub-networks’ flows/streams.

2.a, Scenarios in the “draft-ietf-detnet-ip-ov= er-mpls” and “draft-ietf-detnet-mpls-over-udp-ip”. I see = “mpls-over-ip”

in the flow type next hop choices, but nothing regar= ding “ip-over-mpls”. Have I missed something?

[Don] I’m finding it hard to parse your meanin= g - Sorry. If you are referring to the YANG yes we have ip-over= -mpls

This is well specified. We do not have MPLS ov= er IP but we have aggregation of Service Sub-layer and Forwarding Sublayer = over

A service sublayer which could be IP. We do no= t explicitly support IP-IN-IP tunnel formats (additional field). &nbs= p;

We need WG guidance if this is to be done in this YA= NG or other. We made a call to have a clear aggregation model and

these needed to be implemented first.

2.b, Further, I have had also the TSN drafts in mind= “draft-ietf-detnet-ip-over-tsn”, “draft-ietf-detnet-mpls= -over-tsn”

and “draft-ietf-detnet-tsn-vpn-over-mpls”= ;. YANG for TSN is defined in IEEE, however for mapping DetNet flows

to/from TSN Streams may require e.g., some common id= entifier. What do You think?

[Don] We allow Ethernet to be included from an appli= cation as an interface. This is a transparent data service.

We can prune that stream on Ethernet parameters only= egress we simply hand the decapsulated stream back to the application.

However if you create an application interface for a= second stream you can map that to a different service sub layer and

Use the service sub-layer to differentiate the appli= cation at egress.

Both of these issues need additional discussion if t= he WG if we are to add the function.

The configuration and aggregation model provid= es some capabilities because they are building blocks.

Cheers

Bala’zs

From: Don Fedyk <dfedyk@labn.net>
Sent: Wednesday, March 3, 2021 3:10 PM
To: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>; 'DetNet WG' <detnet@ietf.org>
Subject: RE: New version of DetNet YANG

Hi Bal=E1zs

We have the traffic flow requirements and flow-spec requests that can= be applied at the application, service sub-layer and forwarding sub-layer.= Each forwaring type, IP and MPLS have the data plane QoS objects. ( DSCP for IP and traffic-class for= MPLS).   An SDN controller or a management plane could correlate= QoS reservation of resources to a flow with the appropriate markings. &nbs= p;And you could stich a path on various nodes with MPLS label. But we do not have a control plane function to correlate QoS t= raffic in the YANG models that we have to any database. (i.e. a traffic eng= ineering database)   I suspect there are existing YANG models tha= t do some of that but we have not looked into that because we do not have DetNet documents in this area.
I’m not sure what you mean by sub-network specific YANG parame= ters. Is there a DetNet document that you are referring to?

Cheers

Don

From: Bal=E1zs Varga A <balazs.a.varga@ericsson.com>
Sent: Tuesday, March 2, 2021 9:18 AM
To: Don Fedyk <dfedyk@labn.net= >; 'DetNet WG' <detnet@ietf.or= g>
Subject: RE: New version of DetNet YANG

Hi Don,

Thanks, cool.

Two question for clarification (we may discuss on Mo= nday as well):

1, Are there YANG parameters defined for the QoS fun= ctions used in the DetNet

forwarding sub-layer? I have found e.g., the forward= ing sub-layer related MPLS

label operations, but not the QoS related ones.=

2, Are there sub-networks specific YANG parameters d= efined/referenced as well?

Many thanks

Bala’zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Don Fedyk
Sent: Friday, February 19, 2021 8:37 PM
To: 'DetNet WG' <detnet@ietf.o= rg>
Subject: [Detnet] New version of DetNet YANG

Hi

We have posted a new version of the DetNet YANG=

https://www.ietf.org/archive/id/draft-ietf-detnet-= yang-11.txt

(This issue corrects some yang validation checks that a few days = earlier submission missed. )

This version the authors think is at a level of completeness we w= ill be asking for WG last call.

Please have a look and any post comments. We plan to presen= t the highlights at the next DetNet meeting.

Cheers,

Don

--_000_MN2PR14MB40300A79231A97657D1F0A11BB939MN2PR14MB4030namp_-- From nobody Mon Mar 8 06:29:08 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3782F3A2B25 for ; Mon, 8 Mar 2021 06:29:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=labn.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sdyl5aWkRlwB for ; Mon, 8 Mar 2021 06:29:04 -0800 (PST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2101.outbound.protection.outlook.com [40.107.243.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC4DA3A2B22 for ; Mon, 8 Mar 2021 06:29:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q1ZcODX4GS3d6fjGtL8hIO4XrlMrx2XKd0MKg1XZrpM33zCV2ageAPW0DZGXGPwY9OaG7MreRhv+DuouBeC8sUWpcszgsKsY2tiuin8bXTwtjxhYvVElxtGdAzGP9rbFqeZoJT6VNwglvgZKBcky1r7FehPThTofXUvWtm7x305hMwoMWTmSxKvZF14oXQAZat9F93kWG83RaoFseF403CQZxczG2r6utfUqlvcBW3l9IHkErnkCG5OKx9o/grxQISO6LRPSAuLX67GC+21BFCa5f12ovh4R9ehp1Z6PRaw2V0PjOBh0okPD9DTCsLpPZjLZ56FkadqlMu3zCzwAQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hSl9kKG6VHYP6BMFhqQjmVVcGCAX63Q5vI3rsH92awY=; b=Jk1n7X+LOfyYCKOojjMtbsF+DiXDzvfKyg+rHzJEPYCXeAOW3vF09w2FuLRjqISeFBNseDiArK0xLjJJ6acnkPDMB4FdVWgmQ2lo2tQLalGoRIU3p5AUyPC76qti+eORK7NLxBjk2ss/xHq0WW04bp1Vh5JCf1Y3ho/ocCvtI39nTmd7QeAwvk7IGQAkvgHkLkyoSknsfX1euyi5GNLVmnjM0Y0hMUA02kYgQbw0JABZvfgpvNt2+M+mXTcyzVtNosN3Mk3N+7DAdRyF4BKtFpQuEqvGQxwoyuqMuX2baA/v7am5HFNXrbAcIAH4OMgN33DTgk/oLUNhFdBjAuKXHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=labn.net; dmarc=pass action=none header.from=labn.net; dkim=pass header.d=labn.net; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=labn.onmicrosoft.com; s=selector2-labn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hSl9kKG6VHYP6BMFhqQjmVVcGCAX63Q5vI3rsH92awY=; b=usuvAbkpUUhbQGARebHn6pQmyzjDiwNKRvic/rpR6kYiS70sQDXvKvR8PShu/vxEhSxoj88xoY4OyGhYCxnYEkC68mAvBXFL+lfQTWQV5deaN3oQsS51mQaorVyTuKucQomuUZkLyFzw/zbEGjue+AIDi/6GNIDjxGno/aqVwjc= Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=labn.net; Received: from BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) by MN2PR14MB3423.namprd14.prod.outlook.com (2603:10b6:208:1a2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.23; Mon, 8 Mar 2021 14:29:03 +0000 Received: from BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b]) by BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b%7]) with mapi id 15.20.3912.027; Mon, 8 Mar 2021 14:29:03 +0000 To: DetNet WG From: Lou Berger Message-ID: <69217591-0d15-63a7-1534-69c7a4db6aee@labn.net> Date: Mon, 8 Mar 2021 09:29:02 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [100.15.108.238] X-ClientProxiedBy: BL0PR02CA0010.namprd02.prod.outlook.com (2603:10b6:207:3c::23) To BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [127.0.0.1] (100.15.108.238) by BL0PR02CA0010.namprd02.prod.outlook.com (2603:10b6:207:3c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Mon, 8 Mar 2021 14:29:02 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3e1c457d-0836-4340-913b-08d8e23e858b X-MS-TrafficTypeDiagnostic: MN2PR14MB3423: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Mbi7tYnA/79bAC/UnzIFDFh7907Fl/oiWgxndipN02Xkx9POHYS7361bGWdTbG+qBws1yp2aZ6HdAuLfbIP7p5eiLXpWnikfNzijUNfMXKG6de3BWDBODJgkZOWxWmkacfHA3KKy/yakgqj3KwauUofuLGevPnvxR3u9pD5DPkJlOtQWK4pBFplWP6XSxHYIe7ZSzl714GnUyef0aMiRsN+St9nvNKZJ1bLhbAXqYpZhMnewRYjXmt5rDjV4jwbW4JcLVhQzfoiltWZlBw+UYTo2kBQzdMC22PS0l6XN0ogrsKAPZ2MMZrGSRgXeSirTCUNHtQO2vj64fViIfrROGY3tTVEakyuODRpRcKYj2ao4272TS+kZ2Hrz/+M+sxUzySjxVXBkUsYy4YCRuuGF0O2cF2cTbFdQ0FmSWfNNUyhS3l7N09GHjokM4GHUeX5XMWFDz5DGsASmHvQklQrVbA8t2VIu8bVHSf+qbQzT+BsvucVmwdofpBJF1+B3yrvDmj3oXINOSf8uxdxY69dE+UT+WpP+2QjypjMgzvRHM4mXiLp//WPb6yYP+KXgPL9BCufZc0XDpdbZ8H68p3GD9MVY8keeNjbaiL00lGCWzX2E1hTjnNGh6j0y/Vy0MFx4ydEGaRJ9jTOzrLVSSchMjRMP1ylhJ7HQqB+Dckv2mlZvvxUE0ms9K5lVvbYTPRcO2ccAVDOAtJ7gdM2sQCjLeQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR14MB3779.namprd14.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(376002)(346002)(396003)(136003)(39830400003)(36756003)(478600001)(966005)(8676002)(45640500001)(31696002)(7126003)(956004)(2616005)(66556008)(86362001)(6916009)(66946007)(66476007)(26005)(186003)(16526019)(31686004)(316002)(16576012)(52116002)(8936002)(83380400001)(2906002)(7116003)(6486002)(7246003)(4744005)(19273905006)(3480700007)(5660300002)(45980500001)(43740500002)(563064011); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?WkgzbFRld0kwbDdsYkxENFcraXVZS3FnM2NwM1p4eSs1WTFLbFNKTjBCUVd3?= =?utf-8?B?RWVvTmo4bjd3MmFreUhKcnNZazE4Ukg4a2M4cmtEQWVLUHEvbWZzLzZ5YUlQ?= =?utf-8?B?Ky9zV1pFamlEcFFwK2tCbU80ZTlHWXBMbjFSNHV3V3RDVDRlNFJ5YVFUV2Ry?= =?utf-8?B?TGNtZkNQdm1VbUp1azZjMnUzT1R0ZGtXUlFOWmtpcnNvVFpHenpFMFliYWVJ?= =?utf-8?B?WXVyMENFRTdGTG95K1RDeEx0UUNZVEcyTVgrMGFmcThjQkNRTldyUmZ5cHJw?= =?utf-8?B?bmZjSmZJNDg3OHFNZGpWMVBkMWF5bVVoQkQ4cUc1dzBHVFlOMWpvZG1Ga0NE?= =?utf-8?B?YmNxUkZHaS90QWRaMm1qWGpaQVhCOTlOYlBaV1NLY1dYQ0hlNWtTTnpsTDZW?= =?utf-8?B?UDR0cSs4Rm8zb0NpR3h4VElRVVhQdDh6WndwSW8rRHR3S2lxd3d2Ry9XMkMz?= =?utf-8?B?Q3MxdS9tSVQ1WU0xeVdkcWhwd3UvTmxkdWV3R1o0YXVZTUxoTHhQS3o1bHFR?= =?utf-8?B?QU05MkE1TndIZS9iNzVTTGlzMUJtNVJ1eVNSelprbFBIN2dlKzNON0FUejFj?= =?utf-8?B?MmNUMG5LbHNWK3hNbjlJazZsbmhjL2lidEM2SjcwS3ZiKzh2UEhncGJiNDR6?= =?utf-8?B?a0FrQUg2T0VpTTZJcVhTN1RscmJUYmlnZFdpTW1CRzY3L3QvQktZdm1ZLzBo?= =?utf-8?B?NTdkRkd6OFkrUFJwdkhDbmxCdnhxMGM1TjlQck1Ud0pmTWIwYUlYdzFKMkVP?= =?utf-8?B?ZVVLZTltNmlQRTl4SytYMjZTdUNGczJybkdLOTJLT1dkSGRMM2M4bkFYeXIz?= =?utf-8?B?WlpOOStBbXBnTklmdURzQTRwMSs5MGpTVEkxc3NGTElNSnFKZ2lFeHFsK25N?= =?utf-8?B?VURNK29LTURrS0F1dkxadHJIeHRoZ3loV0JrUE90RFRDU0dUckRqZFFEUitP?= =?utf-8?B?b2JXbzNuMWkzbExvTmplKytrRFdZRW8xRE9CRHUva1ZlaklWSkRYaUVMczZN?= =?utf-8?B?UUh2NFFvbXlKSEtjTUxmTWxVRmk0bGNWYzFYOTRSTXJodThZN1pBK3R0dUdl?= =?utf-8?B?MmY0aTNDaXhHQ005TlBRbzgwd1Z0bncwZlIxL2s1VjMxenBIVHhacmxGcnU2?= =?utf-8?B?SkRwQll0NithSW5yV3o1UUFZMGtvQjhhTk1DUXBWRERZVmdEV3RXRFhFdzdP?= =?utf-8?B?eXNKUDBpSEhoQXVGQ3l0Y1NoZzc4djYydnV0S2tNUnZrYzl1ek55UHlFT3pm?= =?utf-8?B?eUc3c1p4QVkxSVcvZUVPVEhoRWpFMlNYUmNybHZxY0tqTU00bHNpNXBteVJZ?= =?utf-8?B?Z2pVNjlxcFRjZWxpU2dWSUZrNlViT0ovWHFSRWtxYy9OK0dhckp2Z3pyRi9G?= =?utf-8?B?M24waVMxb2FuK202U2tYSFl1RjkyZGVncFptRGNjK3ZueitodDBMZmM1ODhF?= =?utf-8?B?UEZPMDY1cDlvWTUxZWJ5a3QxMzNpTEhqMHVGcEpvd2hpTmVlKzM5UDJZaTEx?= =?utf-8?B?Mkd5MWRaeVhQWEdGVHZaMzNublNPSUowMDlSSEU2clRPNlppbXU5MmlRWHRz?= =?utf-8?B?OGdaQkZNZ05LS3FWcmdhQzl2UHRiYm1IVXo3N3V3S2E5bTNpdWY0Y3pLTndj?= =?utf-8?B?Rkl0OUFvSkZHVytNMlpXTmMxQ0RBNmJCRUpUY09jK1VYYXpEV2x0MHFpL3ZL?= =?utf-8?B?dmFkNHNlTnhJVE9pNDROT3k5dVc4cEcxVzEyNnRrbjNWNU94NVp1cUY1NEhi?= =?utf-8?Q?yVZ6JEJMxOTTUkdjJig0hVwwxQuQCEHeFcroi21?= X-OriginatorOrg: labn.net X-MS-Exchange-CrossTenant-Network-Message-Id: 3e1c457d-0836-4340-913b-08d8e23e858b X-MS-Exchange-CrossTenant-AuthSource: BL0PR14MB3779.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2021 14:29:03.0625 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: eb60ac54-2184-4344-9b60-40c8b2b72561 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7MofP+4gE2K/C3SmZKRj7pc/4YZHcbxnHgpKikjrR+ojKkPtsgxm5nmxIAXrHEBAygu1N3eOtGKKv/RDTR8W4w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR14MB3423 Archived-At: Subject: [Detnet] session info X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 14:29:06 -0000 Reminder, we're about to get started for our first session... Materials: https://datatracker.ietf.org/meeting/110/session/detnet Note taking:    https://codimd.ietf.org/notes-ietf-110-detnet Jabber:    http://jabber.ietf.org/logs/detnet WG ICS: https://datatracker.ietf.org/meeting/upcoming.ics?filters=detnet Session ICS: https://datatracker.ietf.org/meeting/110/session/28645.ics Meetecho: https://meetings.conf.meetecho.com/ietf110/?group=detnet&short=&item=1 Audio stream: http://mp3.conf.meetecho.com/ietf110/detnet/1.m3u From nobody Mon Mar 8 06:30:58 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F133F3A2A04; Mon, 8 Mar 2021 06:30:50 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.23 X-Spam-Level: X-Spam-Status: No, score=-1.23 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, INVALID_MSGID=0.568, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1TgOduNg7PGN; Mon, 8 Mar 2021 06:30:48 -0800 (PST) Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 002983A29DD; Mon, 8 Mar 2021 06:30:47 -0800 (PST) Received: from fraeml701-chm.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DvLFw4GRFz67wmP; Mon, 8 Mar 2021 22:24:52 +0800 (CST) Received: from dggeme754-chm.china.huawei.com (10.3.19.100) by fraeml701-chm.china.huawei.com (10.206.15.50) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Mon, 8 Mar 2021 15:30:39 +0100 Received: from dggeme751-chm.china.huawei.com (10.3.19.97) by dggeme754-chm.china.huawei.com (10.3.19.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Mon, 8 Mar 2021 22:30:38 +0800 Received: from dggeme751-chm.china.huawei.com ([169.254.210.30]) by dggeme751-chm.china.huawei.com ([169.254.210.30]) with mapi id 15.01.2106.013; Mon, 8 Mar 2021 22:30:38 +0800 From: "Liubingyang (Bryan)" To: Yaakov Stein , Haoyu Song , detnet , spring , pce Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpAAA9T+5AAAZ19cAAB5aG+ Date: Mon, 8 Mar 2021 14:30:37 +0000 Message-ID: F09AC6CE-AECC-4EC0-B0FA-34EB7844EDE2 References: <9b01579697ba42cf90b0bd5861730fc3@huawei.com>, In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_F09AC6CEAECC4EC0B0FA34EB7844EDE2_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 14:30:51 -0000 --_000_F09AC6CEAECC4EC0B0FA34EB7844EDE2_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Some advantages. If you use budget instead of deadline, you don't need accurate time sync. If you use budget, the time budget encoded in packet require fewer bits tha= n using absolute time. The tightness is related to whether and how many flows can be admitted. It = is related to network resource utilization and revenue. If e2e deadline is 1ms, then time sync with 1ms accuracy will not work well= . Using budget helps resolve this problem. Just my two cents. Bryan From:Yaakov Stein To:Liubingyang (Bryan) ;Haoyu Song ;detnet ;spring ;pce Date:2021-03-08 21:38:20 Subject:RE: new draft on segment routing approach to TSN Bryan The local deadlines are specified in absolute time =96 not deltas. So there is no need to add the time saved =96 it is already there! As I previously replied, the accuracy of the time sync depends on the tight= ness of the budget. It has nothing to do with the encoding. Y(J)S From: Liubingyang (Bryan) Sent: 08/03/2021 14:55 To: Yaakov Stein ; Haoyu Song ;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Great work. One suggestion. If we divide the end-to-end delay budget to per-hop budgets, and carry them= in SR header, we can mimic using the end-to-end absolute deadline. One thi= ng we need to do is, when the actual time used in a hop is less than the bu= dget of this hop, you can add the remaining budget to the budget of next ho= p. In this case, you can save a lot of bits used for time, and you don=92t = have to rely on accurate time sync. Bryan (Bingyang Liu) From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Monday, March 8, 2021 1:37 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We=92ll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song = > Sent: 08/03/2021 04:41 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I=92ll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn=92t be highly accurate =96 it depends on the tightn= ess of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I=92ll leave the OAM for later. However there are already many high accurac= y performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it=92s vio= lated. Other independent methods are possible, but it=92s better to conside= r if it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I=92ve heard of PIFO (Push In First out) but not PIPO. Is this a typo or so= mething new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that=92s a typo. I mean PIFO (although we do have a paper under re= view using the name PIPO). Yes I agree those are just abstract primitives. = The actual implementation, if customized to a particular algorithm, would b= e simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I=92ll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I=92ll let the chairs decide where discussions should be held. Y(J)S --_000_F09AC6CEAECC4EC0B0FA34EB7844EDE2_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Some advantages.

If you use budget instead of deadline, you don't need accurate time sy= nc.

If you use budget, the time budget encoded in packet require fewer bit= s than using absolute time.

The tightness is related to whether and how many flows can be admitted= . It is related to network resource utilization and revenue.

If e2e deadline is 1ms, then time sync with 1ms accuracy will not work= well. Using budget helps resolve this problem.

Just my two cents.

Bryan

From:Yaakov Stein <yaakov_s@r= ad.com>

To:Liubingyang (Bryan) <liubi= ngyang@huawei.com>;Haoyu Song <haoyu.song@futurewei.com>;detnet &l= t;detnet@ietf.org>;spring <spring@ietf.org>;pce <pce@ietf.org&g= t;

Date:2021-03-08 21:38:20

Subject:RE: new draft on segment= routing approach to TSN

Brya= n

&nbs= p;

The = local deadlines are specified in absolute time =96 not deltas.

So t= here is no need to add the time saved =96 it is already there!

&nbs= p;

As I= previously replied, the accuracy of the time sync depends on the tightness= of the budget.

It h= as nothing to do with the encoding.

&nbs= p;

Y(J)= S

From: Liubingyang (Bryan) <liubingyang@hua= wei.com>
Sent: 08/03/2021 14:55
To: Yaakov Stein <yaakov_s@rad.com>; Haoyu Song <haoyu.song= @futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Hi Y= aakov,

&nbs= p;

Grea= t work. One suggestion.

If w= e divide the end-to-end delay budget to per-hop budgets, and carry them in = SR header, we can mimic using the end-to-end absolute deadline. One thing w= e need to do is, when the actual time used in a hop is less than the budget of this hop, you can add the remaini= ng budget to the budget of next hop. In this case, you can save a lot of bi= ts used for time, and you don=92t have to rely on accurate time sync.

&nbs= p;

Brya= n (Bingyang Liu)

&nbs= p;

From: detnet [mailto:detnet-bounces@= ietf.org] On Behalf Of Yaakov Stein
Sent: Monday, March 8, 2021 1:37 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Haoyu

I think we are in agreement.=

I do not see the need for explicitl= y handling the case of a packet missing a LOCAL deadline,

since the following switches will a= lready handle this case optimally (and may still meet the overall budget).<= /span>

Counting packets that miss their de= lay budget is indeed important,

and a counter could be configured i= n the egress router for this.

We=92ll need to define this when we= get to the protocol specification.

It would be advantageous to put a t= hreshold on the failure rate

and feed this back to the path/stac= k optimizer.

Y(J)S

From: Haoyu Song <haoyu.song@fut= urewei.com>
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Hi Yaakov,

Some feedback inline.

Best regards,

Haoyu

From: Yaakov Stein <yaakov_s@rad.com= >
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Haoyu,

I=92ll address your points:<= /p>

> The use of clock time as deadl= ine requires network synchronization

That is a basic assumption of TSN n= etworks (IMO the defining assumption).

However, the sync needn=92t be high= ly accurate =96 it depends on the tightness of the delay budgets.
>> Yes, I understand it. What= I mean is that the method mentioned in the draft seems to be also applicab= le to other types of networks. For example, we can envision some time-criti= cal traffic in DCN or WAN. If a protocol would be developed, can it also serve the other networks? If so, it would be bet= ter.

> accurate measurement of per-li= nk propagation time

Yes, I am assuming that once an for= all someone does a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measuremen= t of the links, and these are stored in some database.

Once again, the required accuracy d= epends on the delay budgets.

> which can somehow limit the ap= plication scope of this work

If the delay budget only above the = physically minimal delay by say less than 100 microseconds,

I agree that the previous two issue= s MUST be carried out. But in such cases there is no alternative.
If the delay budget is much higher = than that, then one could use an RSVP-like mechanism,

sending a packet (or several packet= s) from source to destination collecting a stack of timestamps,

and then using that stack for the f= ollowing packets.

> Mechanism should be provisione= d to track where the timing requirement is violated and by how much<= /p>
I=92ll leave the OAM for later. How= ever there are already many high accuracy performance measurement technique= s and protocols for this.

>> For this I mean something = recorded in the same packet with the deadlines. If it misses the deadline, = the receiver may need to know where it=92s violated. Other independent meth= ods are possible, but it=92s better to consider if it can be integrated in the current proposal.

> Recently programmable schedule= r research has proposed several primitives

Yes, I tried to stress that this ID= is not limited to EDF (although sometimes that is a good strategy).=

One can even reproduce Qbv behavior= using a stack of deadlines (although why would one wish to do so?).=

> such as PIPO and PIEO
I=92ve heard of PIFO (Push In First= out) but not PIPO. Is this a typo or something new?

I agree that there are mechanisms t= hat are optimized for hardware, but I have come up with a very nice hardwar= e implementation for PEDF

and prefer to find hardware impleme= ntations for optimal schedulers, rather than to determine schedulers based = on optimal hardware.

>> Sorry that=92s a typo. I m= ean PIFO (although we do have a paper under review using the name PIPO). Ye= s I agree those are just abstract primitives. The actual implementation, if= customized to a particular algorithm, would be simpler.

Y(J)S

From: Haoyu Song <haoyu.song@fut= urewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

CAUTION: External s= ender. Do not click links or open attachments unless you know the content is safe.

Hi Yaakov,

Just got a chance to read your draf= t. I agree with the comments of the others that this is a very interesting = work. I=92ll just add a few points.

The use of clock time a= s deadline requires network synchronization, and accurate measurement of pe= r-link propagation time, which can somehow limit the application scope of t= his work. Alternatively, one can simply budget a device latency which require a router/switch to obey. In case the= overall budget is evenly divided by the hops, a single parameter is enough= . Of course, if one wants to customize the budget on each hop (which might = be necessary considering the different capability/capacity of each hop), a stack is still needed.
Mechanism should be provis= ioned to track where the timing requirement is violated and by how much (e.= g., using timestamp or flag). This would be very useful for troubleshooting= .
Recently p= rogrammable scheduler research has proposed several primitives such as PIPO= and PIEO and provided feasible hardware implementations. The scheme propos= ed in this draft can easily fit into these primitives.

Best regards,

Haoyu

From: spring <spring-bounces@ietf= .org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN

All,

I would like to call your attention= to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-based= approach (similar to segment routing) to time sensitive networking.=

It furthermore proposes combining s= egment routing with this approach to TSN

resulting in a unified approach to = forwarding and scheduling.

The draft is information at this po= int, since it discusses the concepts and does not yet pin down the precise = formats.

Apologies for simultaneously sendin= g to 3 lists,

but I am not sure which WG is the m= ost appropriate for discussions of this topic.

DetNet is most relevant= since the whole point is to control end-to-end latency of a time-sensitive= flow.
Sprin= g is also directly relevant due to the use of a stack in the header and the= combined approach just mentioned.
PCE is relevant to the case of a central server join= tly computing an optimal path and local deadline stack.

I=92ll let the chairs decide where = discussions should be held.

Y(J)S

--_000_F09AC6CEAECC4EC0B0FA34EB7844EDE2_-- From nobody Mon Mar 8 08:32:18 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7FB153A0ACB for ; Mon, 8 Mar 2021 08:32:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=herbertland-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d4WkNCyACUjU for ; Mon, 8 Mar 2021 08:32:12 -0800 (PST) Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F37F33A0BCC for ; Mon, 8 Mar 2021 08:32:11 -0800 (PST) Received: by mail-ej1-x629.google.com with SMTP id bm21so21594147ejb.4 for ; Mon, 08 Mar 2021 08:32:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herbertland-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=HvbF55KUklxIfSy9p2wQ0v2raKsJdy8fQIb37tsDm0s=; b=mimTXS5QBDeEWkVJVicGvWatp4guqltITMMkyggkuEVABC6oOiGWYRiKBE8mPvwdob V0mMkKn32mu1CAz8QBHMwwAFEvxKR9RgjXhRUrioNzVqQj9pOlvtUp0m4dq6iAusmiwQ A4n4azIa0WW0DvmtcdGX55Q7hpXs8Xwmg2aY0OA4UiXNfl16oQi3adxoZoGBY1FC8jb1 ndkGanDySSi6b/b4JLqqcdo0RJIFHY2f6JH+wrIsCTWYCOmgNRrWnTyoh0XBzKxR2aGa tcUZNGNeFhvEAynr86vKpgXTv2dIooL5QJqUW7s1pi91odKOjvClpt+hCSt4+mqsTFp7 VqNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=HvbF55KUklxIfSy9p2wQ0v2raKsJdy8fQIb37tsDm0s=; b=kIugwTt6FTtxRULWi1kPXLU6un/0RZ+ACbtVo+dN1MWDhuyTJn73DOoHXCDYoqOoI1 Fixh8MrhBOBVjmPyqrhL1IAtofjm52AnR65U9q6BZNbWj20VrgzRz69W56xBOd662Q/w FGhT9O0iX68rPdh0N6Z+L9Mol/DmcDm3YU9cyk9pLHXZgPVIrybdAQXjTBx1XytppTch fiDzhI/CmOKZe7dc3tFxk0jMIcZWkYJXpZHUJ+GJiFhrYE29ZA47lf2umpUav5jmaZ5p YarKYFBM3vlM3dnGrs9eGrbOgwwA3gXAx0Qfho9qXQHdiPhg/7eAP3rtVWQI6qTcJ2YO wtcg== X-Gm-Message-State: AOAM530FqyKkaHKkoPEzAe+cvcCpmsJ/lyxLVwahX4lE4v+OfvhqP+d1 ZcP1ATYTx2fkBEdbvB5UA+mCC1q/IuBwd8e9flmQew== X-Google-Smtp-Source: ABdhPJyzWlPc5AAkg98Cs9GXOwh3YnF8ovSkqwH+j4fd2Dw1tg177wNjVXLLd12T5IluXbgrqptv2KlyAwvu+YxGxE4= X-Received: by 2002:a17:906:4146:: with SMTP id l6mr16228243ejk.295.1615221129524; Mon, 08 Mar 2021 08:32:09 -0800 (PST) MIME-Version: 1.0 References: <3fdc1006788e47e59cfb8dcc03e9bce6@huawei.com> In-Reply-To: <3fdc1006788e47e59cfb8dcc03e9bce6@huawei.com> From: Tom Herbert Date: Mon, 8 Mar 2021 09:31:58 -0700 Message-ID: To: "Yangfan (IP Standard)" Cc: Greg Mirsky , "draft-geng-6man-redundancy-protection-srh@ietf.org" , DetNet WG , 6man WG Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Detnet] Flow Identification in IPv6 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 16:32:14 -0000 On Mon, Mar 8, 2021 at 2:38 AM Yangfan (IP Standard) wrote: > > Hi Greg, > > > > Literally speaking, IPv6 Flow Label could be used to identify a specific= flow needing redundancy protection in SRv6 data plane. But I may have conc= erns that flow label cannot be protected to be unmodified en route. A modif= ied flow label can be a disaster for the traffics which are seeking for de= terministic forwarding, not only this flow, also affecting other flows usin= g redundancy protection. And with several security issues mentioned in RFC6= 437, I doubt if it is a good idea to user IPv6 Flow Label. > > Just my 2cents opinion, how do you and other people see it? > If this is to be used in a SRv6 domain which is itself a limited domain, then I think the problems you mention aren't as much of a concern since flow label would be used in a controlled environment. The upside of using flow label is that it's already in the IPv6 header, it can be consumed by non-SRv6 devices, and putting the same information in TLVs incurs the overhead and cost of processing TLVs in the critical datapath. Tom > > > Regards, > > Fan > > > > > > > > > > =E5=8F=91=E4=BB=B6=E4=BA=BA: Greg Mirsky [mailto:gregimirsky@gmail.com] > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2021=E5=B9=B43=E6=9C=887=E6=97=A5 4= :20 > =E6=94=B6=E4=BB=B6=E4=BA=BA: draft-geng-6man-redundancy-protection-srh@ie= tf.org > =E6=8A=84=E9=80=81: 6man WG ; DetNet WG ;= Greg Mirsky > =E4=B8=BB=E9=A2=98: Flow Identification in IPv6 > > > > Dear Authors, > > thank you for bringing your proposal to the discussion. I agree with your= view that the explicit routing enabled by SRv6 creates an environment wher= e PREOF can be used. And, as we know, The PREOF may be used in a DetNet dom= ain to lower packet loss ratio and provide bounded latency. > > After reading the draft, I've got a question for you. What do you see as = the difference between the IPv6 Flow Label per RFC 6437 and the Flow Identi= fication field in the TLV proposed in the draft? Could the IPv6 Flow Label = be used to identify the flow for the PREOF? > > > > Regards, > > Greg > > -------------------------------------------------------------------- > IETF IPv6 working group mailing list > ipv6@ietf.org > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 > -------------------------------------------------------------------- From nobody Mon Mar 8 08:36:33 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDB113A0D07; Mon, 8 Mar 2021 08:36:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.517 X-Spam-Level: X-Spam-Status: No, score=-1.517 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URI_DOTEDU=0.132] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aL9Mc-O7kz67; Mon, 8 Mar 2021 08:36:29 -0800 (PST) Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [131.188.34.40]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 67D0D3A0CCF; Mon, 8 Mar 2021 08:36:29 -0800 (PST) Received: from faui48f.informatik.uni-erlangen.de (faui48f.informatik.uni-erlangen.de [131.188.34.52]) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTP id 984BD54804B; Mon, 8 Mar 2021 17:36:23 +0100 (CET) Received: by faui48f.informatik.uni-erlangen.de (Postfix, from userid 10463) id 92209440166; Mon, 8 Mar 2021 17:36:23 +0100 (CET) Date: Mon, 8 Mar 2021 17:36:23 +0100 From: Toerless Eckert To: Yaakov Stein Cc: "detnet@ietf.org" , Haoyu Song , "spring@ietf.org" , "pce@ietf.org" Message-ID: <20210308163623.GA58143@faui48f.informatik.uni-erlangen.de> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Archived-At: Subject: [Detnet] PEDF / PIFO / ... (was: Re: new draft on segment routing approach to TSN) X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 16:36:32 -0000 > Yaakov wrote: > I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or something new? > I agree that there are mechanisms that are optimized for hardware, but I have come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather than to determine schedulers based on optimal hardware. PEDF ? There was a long debate in the congestion control technology in TSV about the scalability issues of flow-aware AQM mechanisms and AFAIK, ultimately, the AQM mechanisms that won out where the ones that did not require this (such as PIE). We also had round 1 of deterministic services (if i may call them that) using rfc2212 with RSVP fail because of real or assumed infeasibility to scale on multiple dimensions. IMHO it actually failed also because of assumed infeasibility becasue there was no good analysis and documentation of what actually could be made to work (badmouthing...). >From this experience i can only recommend to make sure that we do understand what and how something is feasible to be implemented a what scale and speed. For example, i am not aware to have seen general purpose EDF hardware at scale. But i am very interested for any pointers. On the research side, this one is the oldest one for PIFO i know: http://web.mit.edu/pifo/pifo-sigcomm.pdf [ i am of course mentioning PIFO because by using deadline as the PIFO rank, one has a simple approach to implement EDF]. In this work, if i remember my analysis correctly, the scale still depends on the number of flows and the ability to identify packets to flows (need to read it again though). This _may_ be acceptable in specific use-cases but should IMHO be well understood and documented, especially when the view from the outside is that by using e.g.: SR packet headers it is "looking" as if there is really no per-flow scaling aspect to the hardware requiremens. After all, the idea of source routing with SR and having the state in the packet is to eliminate the need to have) and scale it in the router. Cheers Toerless > >> Sorry that's a typo. I mean PIFO (although we do have a paper under review using the name PIPO). Yes I agree those are just abstract primitives. The actual implementation, if customized to a particular algorithm, would be simpler. > > Y(J)S > > From: Haoyu Song > > Sent: 05/03/2021 22:46 > To: Yaakov Stein >; detnet@ietf.org; spring@ietf.org; pce@ietf.org > Subject: RE: new draft on segment routing approach to TSN > > > CAUTION: External sender. Do not click links or open attachments unless you know the content is safe. > Hi Yaakov, > > Just got a chance to read your draft. I agree with the comments of the others that this is a very interesting work. I'll just add a few points. > > > 1. The use of clock time as deadline requires network synchronization, and accurate measurement of per-link propagation time, which can somehow limit the application scope of this work. Alternatively, one can simply budget a device latency which require a router/switch to obey. In case the overall budget is evenly divided by the hops, a single parameter is enough. Of course, if one wants to customize the budget on each hop (which might be necessary considering the different capability/capacity of each hop), a stack is still needed. > 2. Mechanism should be provisioned to track where the timing requirement is violated and by how much (e.g., using timestamp or flag). This would be very useful for troubleshooting. > 3. Recently programmable scheduler research has proposed several primitives such as PIPO and PIEO and provided feasible hardware implementations. The scheme proposed in this draft can easily fit into these primitives. > > Best regards, > Haoyu > From: spring > On Behalf Of Yaakov Stein > Sent: Tuesday, February 23, 2021 5:14 AM > To: detnet@ietf.org; spring@ietf.org; pce@ietf.org > Subject: [spring] new draft on segment routing approach to TSN > > All, > > I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt > which describes using a stack-based approach (similar to segment routing) to time sensitive networking. > It furthermore proposes combining segment routing with this approach to TSN > resulting in a unified approach to forwarding and scheduling. > > The draft is information at this point, since it discusses the concepts and does not yet pin down the precise formats. > > Apologies for simultaneously sending to 3 lists, > but I am not sure which WG is the most appropriate for discussions of this topic. > > * DetNet is most relevant since the whole point is to control end-to-end latency of a time-sensitive flow. > * Spring is also directly relevant due to the use of a stack in the header and the combined approach just mentioned. > * PCE is relevant to the case of a central server jointly computing an optimal path and local deadline stack. > I'll let the chairs decide where discussions should be held. > > Y(J)S > > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://www.ietf.org/mailman/listinfo/detnet -- --- tte@cs.fau.de From nobody Mon Mar 8 20:03:34 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 056333A0DFE; Mon, 8 Mar 2021 20:03:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.099 X-Spam-Level: X-Spam-Status: No, score=-1.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 34JV5-1tcfU2; Mon, 8 Mar 2021 20:03:30 -0800 (PST) Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9B4B03A0DFF; Mon, 8 Mar 2021 20:03:30 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id j6so5909014plx.6; Mon, 08 Mar 2021 20:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=eSvcdBEJOPIsmP95kMC5IFwDDrauSSldzY22FKGjDWM=; b=a/PZ7p4M8ZimL+zWeQ+6JHmqcOXfvrSEqAiVfO0SDUk1Jco6r0BdORyaRZSpZvHObt Fl8rtP0h0A2fWREqFNOLjKwkGjLRjJco+ZW39VF615r4qCRfpd0yhqWUmqWmFZrXOzng +FDX4PoE0Jex/5tzBr1pM6pMXTo2OwLyCwy32ka6ExGUHHB9eqNdNWQMOC9Y565BVP3P L4GO5mK2DFy9xqgv0LWqfXo7rfcAwxXrqIw25nvS1vbYA69dMTggTmwjOTipMRKe5qx8 FRrETEcPh9OLmjAY52ESCVJHS9mWPUkYOkOpue9f6hE0X4lWX+APwLx8TYQFNKGb7OMe hDGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=eSvcdBEJOPIsmP95kMC5IFwDDrauSSldzY22FKGjDWM=; b=afXKEd4DHETrMkOnCm4z2lGTkeflgGrLJ2GNOk37vI82aWO2ymR+vaL3rrwFVGjTbo m4/G/rupnOqEtbIiECHunCkUPxBWN7fqhZBWSHrbuvV6VtBdnJiRMt1PjWeZGUeJu7qM rBpnfWg8FE4fSYmWyrhuTPmxb8VckE7ww0VpB2drz84m9qonnNX86iqbgdkhRJf8w3+6 9FVa7Oxl29E1x8xUqa7VZzzIrG4dzf+8SzPLaHYI6sLn3trf/B+phXAUu7j/bEt8w063 jrZQ4Y9TwY8Vrra8w4L9YI6u8v2uBXvH+5mVnhxW6IsrMssGHSvZaFKd68IKaWfZM5wV hJVg== X-Gm-Message-State: AOAM530pgS65JU9Uv0QcbOOZ1DEaRAy6RI87kK2HmLr/Br68zjCpUy00 CLOAQek2OX1qKn0mKrz7GvzSH1BAJFZBTw== X-Google-Smtp-Source: ABdhPJzS0JunBHUBKxqg/kSvyAmJa89kffqjvkzuGpglHdLpOwJA8Rc34BpmsUXeiqlQBqYoahPfDA== X-Received: by 2002:a17:90a:5510:: with SMTP id b16mr2496867pji.87.1615262608524; Mon, 08 Mar 2021 20:03:28 -0800 (PST) Received: from [192.168.178.20] ([151.210.131.28]) by smtp.gmail.com with ESMTPSA id s27sm10917685pgk.77.2021.03.08.20.03.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Mar 2021 20:03:27 -0800 (PST) To: Tom Herbert , "Yangfan (IP Standard)" Cc: DetNet WG , 6man WG , "draft-geng-6man-redundancy-protection-srh@ietf.org" References: <3fdc1006788e47e59cfb8dcc03e9bce6@huawei.com> From: Brian E Carpenter Message-ID: <9ff8cb12-d23d-74c2-fea7-900d1d4e2974@gmail.com> Date: Tue, 9 Mar 2021 17:03:22 +1300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Detnet] Flow Identification in IPv6 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 04:03:32 -0000 On 09-Mar-21 05:31, Tom Herbert wrote: > On Mon, Mar 8, 2021 at 2:38 AM Yangfan (IP Standard) > wrote: >> >> Hi Greg, >> >> Literally speaking, IPv6 Flow Label could be used to identify a speci= fic flow needing redundancy protection in SRv6 data plane. But I may have= concerns that flow label cannot be protected to be unmodified en route. = A modified flow label can be a disaster for the traffics which are seeki= ng for deterministic forwarding, not only this flow, also affecting other= flows using redundancy protection. And with several security issues ment= ioned in RFC6437, I doubt if it is a good idea to user IPv6 Flow Label. >> >> Just my 2cents opinion, how do you and other people see it? >> >=20 > If this is to be used in a SRv6 domain which is itself a limited > domain, then I think the problems you mention aren't as much of a > concern since flow label would be used in a controlled environment. > The upside of using flow label is that it's already in the IPv6 > header, it can be consumed by non-SRv6 devices, and putting the same > information in TLVs incurs the overhead and cost of processing TLVs in > the critical datapath. The main downside is that it cannot convey any semantics and there is a rule about how its value is created. It's no more at risk of being modified than any other unauthenticated header field, so I agree that within an SRV6 domain malicious modification doesn't seem like a big risk. An attacker who could do that could do any kind of damage they wanted. Regards Brian >=20 > Tom >=20 >> >> >> Regards, >> >> Fan >> >> >> >> >> >> >> >> >> >> =E5=8F=91=E4=BB=B6=E4=BA=BA: Greg Mirsky [mailto:gregimirsky@gmail.com= ] >> =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2021=E5=B9=B43=E6=9C=887=E6=97=A5= 4:20 >> =E6=94=B6=E4=BB=B6=E4=BA=BA: draft-geng-6man-redundancy-protection-srh= @ietf.org >> =E6=8A=84=E9=80=81: 6man WG ; DetNet WG ; Greg Mirsky >> =E4=B8=BB=E9=A2=98: Flow Identification in IPv6 >> >> >> >> Dear Authors, >> >> thank you for bringing your proposal to the discussion. I agree with y= our view that the explicit routing enabled by SRv6 creates an environment= where PREOF can be used. And, as we know, The PREOF may be used in a Det= Net domain to lower packet loss ratio and provide bounded latency. >> >> After reading the draft, I've got a question for you. What do you see = as the difference between the IPv6 Flow Label per RFC 6437 and the Flow I= dentification field in the TLV proposed in the draft? Could the IPv6 Flow= Label be used to identify the flow for the PREOF? >> >> >> >> Regards, >> >> Greg >> >> -------------------------------------------------------------------- >> IETF IPv6 working group mailing list >> ipv6@ietf.org >> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 >> -------------------------------------------------------------------- >=20 > -------------------------------------------------------------------- > IETF IPv6 working group mailing list > ipv6@ietf.org > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 > -------------------------------------------------------------------- >=20 From nobody Mon Mar 8 20:34:35 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E8F93A0F02; Mon, 8 Mar 2021 20:34:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rad365.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OOqduMsxkNta; Mon, 8 Mar 2021 20:34:23 -0800 (PST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50059.outbound.protection.outlook.com [40.107.5.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2FDEA3A0EFC; Mon, 8 Mar 2021 20:34:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T5FO7ITnq/kx4TqegDNjtN/b1Swk+V6wcVEnIydDaJoD+dz+FmkCCzduAE5fwteDsyjZGmCp8hOLr8rO/7CwaQ2huxyX3q722XtrA+CxsPNV7Q2LdU4BdKWTMuihztkfkoE2MTgkjdDvR6aZUzxODhNqEoJTsj/q8Z5slNR4xVCHYW3ntUR8X5fpFxY20U5nVzJfs9AvmeRKUjzUjYH/XhoNNrLaV0oVcmu5FWbNlJZOAf/uPRGAIQl5LkzkS7ixy9FT6/fDSS0a7xkKDoeJl29cNXnLSODW6zKqKdwelHvt0+7Fm4IWZc4yvz0WMVqDhWBvzYMRJQqw/jB2L0XgSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x+c9jBN9FNxUtd3BQQjKJiwg+w+qIxHFm61wAHRT8fU=; b=Pj/TSMl4EumZdK9P+bdsL+DV1RQzjgp/+1FyySaclMVAn6+RybXHrwYopCGiDhBRtu0UAK90NDMuZdtNXRgne/DdPIAsTdCXMhp8uYZbG9rjF8d2k4tuzkr312ZlWDH1zNboxed1Y70TqJ7rsWR1VNyf42OamKxtonArMxdGMFTlz186BSxbCZUJDNje9c5zlqUJtIphT99ayve7/b0ZfhC4mYPXUNTX63hnzJJaITB1CX/cg1B+S08wkDanUyY1PqGhjkyn3dgc6oGC0/O46ovX4C+P0r5Io32+/2F1VhJE2M3/xDzBgtMSNof+bnA/xiw8w0b+1UcLLUNhjuxOWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=rad.com; dmarc=pass action=none header.from=rad.com; dkim=pass header.d=rad.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rad365.onmicrosoft.com; s=selector1-rad365-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x+c9jBN9FNxUtd3BQQjKJiwg+w+qIxHFm61wAHRT8fU=; b=Y0I6AxUXHPBsSZmscbkgoWp08rNBkScv5q2yuQqCWiRFpq8+KqX5ODAoZgYSMQFivQj19U/LYPP3vVwzIu4y9mnrwcPwzCle0UiNjMGiMM8KMa6FStR80vWyT3vHcXCX0+j76u0yQju9FIufDnMFO+nWQxL412jntn+uZWl/fBc= Received: from AM0PR03MB3522.eurprd03.prod.outlook.com (2603:10a6:208:42::23) by AM0PR03MB3969.eurprd03.prod.outlook.com (2603:10a6:208:77::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.26; Tue, 9 Mar 2021 04:34:20 +0000 Received: from AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a]) by AM0PR03MB3522.eurprd03.prod.outlook.com ([fe80::10eb:24f4:1a5e:bc0a%4]) with mapi id 15.20.3912.027; Tue, 9 Mar 2021 04:34:20 +0000 From: Yaakov Stein To: Toerless Eckert CC: "detnet@ietf.org" , Haoyu Song , "spring@ietf.org" , "pce@ietf.org" Thread-Topic: PEDF / PIFO / ... (was: Re: [Detnet] new draft on segment routing approach to TSN) Thread-Index: AQHXFDkx7yt1lj+bJU6wgn6XxpWNAKp7EGYg Date: Tue, 9 Mar 2021 04:34:19 +0000 Message-ID: References: <20210308163623.GA58143@faui48f.informatik.uni-erlangen.de> In-Reply-To: <20210308163623.GA58143@faui48f.informatik.uni-erlangen.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: cs.fau.de; dkim=none (message not signed) header.d=none;cs.fau.de; dmarc=none action=none header.from=rad.com; x-originating-ip: [176.230.181.29] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 27ff31d4-7c2c-4cc8-7ef2-08d8e2b49b4c x-ms-traffictypediagnostic: AM0PR03MB3969: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3968; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ayKNnBVJdxZcp1kzOhTGDYPkVN9Iz5cSRTHTs+M+92MiUjLBl26LZiR5Q4SgIDWHArk4KAAWFbWY5WkkMNE6jNOh2cBwrKfnXch5B2aYHPik8OszDK8ztatwRrs7hORo+3a2QCRCp3dbqleVnLKatrQ+NF1XzdJljU8IxG69ZyAXeYv3GYdWqkOBaijTl+lsOHBQC4C6ZMYyEyUtNlGQnQjA3OKCskHhwEdI52kdum9x4eKu4oK01oTCgv9H0RZ45Wx9wMk12x/Z65JELcR19uzFzvHW7ZVxb2DB0f205hETIIYhA3xJqxSETbJe4WJfBfX6pNmQX77ytDSBXY1hkiYJUAwzeYlFvICu7PGhPFdoTJSWOxXTAFd6Z4TewKDUTer55ep+je+/aEdSKmQXwiEywyK0yzZDHIi3dUPoMpqUoexNFvd18slh1taP7j2iCjbZ0dpqrw5XmxNztZIoayQT+NY2xfbieiy6s39yo9tXmytn4UcyWwp5Ou+DSJWEiaze9Jnq8Q4kBGUdxZ/InF1UQOwddJJyfPbk6zdojUrYlwF7sI/NvIzb3W5oa66cskKBCbUHjX7N+TmVpRab8hp/PeMNZgRFYjaQkbRDRFw= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR03MB3522.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(39850400004)(366004)(346002)(136003)(53546011)(6506007)(55016002)(5660300002)(66946007)(52536014)(76116006)(54906003)(8936002)(9686003)(45080400002)(33656002)(71200400001)(66574015)(2906002)(186003)(966005)(6916009)(86362001)(64756008)(478600001)(4326008)(66476007)(8676002)(66556008)(66446008)(316002)(83380400001)(26005)(7696005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?Hjske8XlQrshIlBmnwApUQxB7u3s+tlHFUnxu9p++rK6ozgtksrGvcOsTeYc?= =?us-ascii?Q?8DRtKZelnclczx/wlWHRZa3Jugv7vkZyW36TvsAzOaohe8pz1ZcBumvJ/EhN?= =?us-ascii?Q?qy3kkKRuBT1X7lCnnUdrDV9DLtsxcKgRzJ9Kh4p/43X0qWEkjCo6vhnewMQw?= =?us-ascii?Q?4F/w6rOoDck+BP5w667BZ0U9TsyB3+FRTt4dSChF6cPfphTWD6VYFrJOVjYJ?= =?us-ascii?Q?MniU5rbweP8Uy6DP0Rhlin7YjHz7UY4o1g5LXvSnM9fBnK+It1QPFZL0fFem?= =?us-ascii?Q?cGV8TOO+EzbtfPQlefJytf1kLHaL2dq2ee64sEDj+Hv6Pg87PI4LEyCAm271?= =?us-ascii?Q?awXAF9558nCEfMaz6PZM9L9lEgnyzNXlsAdx9jxVUparzflhO6wQ4mtpLP3E?= =?us-ascii?Q?i8rEml4UdkdT+bkIgmVlhB0TcTCPxnSjQSyQ2HeZpd7EnNZ6D6um4eftpopi?= =?us-ascii?Q?1B9nRmMyVx/HiDoOvfzPKRUEIY+XO1hGd+mr5zj56nYPWG4N/tlBAs0Nfbze?= =?us-ascii?Q?DfQCPshOOfG8V1oEbXNEePQ+/yZYmyggcZ/+VgIEhJcI9eugsQPbZAdGwV67?= =?us-ascii?Q?UqVmqiN3PSEtmWPDJ9WJjCjzNQORfeWBhNHCdolELfiIBfgAQvXlx85SXOYF?= =?us-ascii?Q?yozGm0j2aZ9MRf6y9wrA7d/ltKq2ISk+lx0BiMqNaSt1h6Be7yrC1Tl49bHJ?= =?us-ascii?Q?dzoNxycyIHuh9EPuKyEIeSqSB3BdVsWEBVB/a8Q6Pqb/UCUJ5W0sa92BtssG?= =?us-ascii?Q?tGnHbGWJ1rdpOzqv56ULLsrq+DiYP9OdsqqfILDfx2A8KSxFvT9CKrK3WObN?= =?us-ascii?Q?evvEDPXE3AFfIzXhhLKfPSwwBruzDF4fLn4s/aKZyO2AytsSPnoP7gzzz7Qa?= =?us-ascii?Q?Ghf3qEDOZY2n5WRpka1WMwEXjkdjQ0qIOYodpyZGpARxFCdTURR4iX3Q+AAQ?= =?us-ascii?Q?K56X5JzjDUYjLO8f+GMLrVQwfehU7iQwfR26iAb0bovT8sgJ5YB5qBF9z73e?= =?us-ascii?Q?4N0Mx3FmZjuTzgES0yRZBrCxstovGkr5BUHWmhVJ6MOo6iz5dsIAQS61Phlr?= =?us-ascii?Q?CwcomAm5M0Sxu7OnwidzJS02uS6UzIAPcYj4ZnkGKurf9TRFvUMYXWQTQJzV?= =?us-ascii?Q?uwE9ZaVGfyAke5QXBssYEgrUuu6LMh9uaivL6XkTroD/spsOV4lhE4+xPFTy?= =?us-ascii?Q?0W6xDE0SWTWyshPSWVutIAGACzdsHTcWBFf2J/twX+uHRbbHGn62Tm32HIzx?= =?us-ascii?Q?O8tLA6xR0p7Vm8vzc5+QUtjbO6ucrlFSqS64yInNUYCxQ35KCAroKN7qr9VC?= =?us-ascii?Q?+FEUYMz0tiqF/PmTql8LviMb?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: rad.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3522.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 27ff31d4-7c2c-4cc8-7ef2-08d8e2b49b4c X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Mar 2021 04:34:19.9568 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f9047108-cc2c-4e48-97a3-43fad1b3bf9d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cJ8rqAI++7WtU8PHNMZUGrxFduW5kyMfOyB0yV/sgtR7e+wo1YwJuCEuUuWkSvaF7OOBjWrr0IspMvc4NRyM1g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3969 Archived-At: Subject: Re: [Detnet] PEDF / PIFO / ... (was: Re: new draft on segment routing approach to TSN) X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 04:34:27 -0000 Toerless I describe PEDF in the ID. It is a twist on the combination of strict prior= ity and EDF. Basically there can be multiple time sensitive flows with different priorit= ies and non-sensitive traffic as well. With PEDF one first looks at the TS structure with the highest priority, however, one doesn't take the packet in the highest priority (like in regul= ar strict priority), but rather you check whether its deadline is closer than the maximum packet= clock-out time from now. If not one checks the next highest priority, with full knowledge that one c= an send another packet and still make it back in time. This can be extended to WFQ as well. Using PEDF loosens the requirements on the optimization. In fact, PEDF and the simple calculation on each flow separately usually outperforms EDF and a more sophisticated optimization calculation. (I don't yet have an analytical explanation of this...) Y(J)S -----Original Message----- From: Toerless Eckert =20 Sent: 08/03/2021 18:36 To: Yaakov Stein Cc: detnet@ietf.org; Haoyu Song ; spring@ietf.org= ; pce@ietf.org Subject: PEDF / PIFO / ... (was: Re: [Detnet] new draft on segment routing = approach to TSN) > Yaakov wrote: > I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or so= mething new? > I agree that there are mechanisms that are optimized for hardware, but I = have come up with a very nice hardware implementation for PEDF and prefer = to find hardware implementations for optimal schedulers, rather than to det= ermine schedulers based on optimal hardware. PEDF ? There was a long debate in the congestion control technology in TSV about t= he scalability issues of flow-aware AQM mechanisms and AFAIK, ultimately, t= he AQM mechanisms that won out where the ones that did not require this (su= ch as PIE). We also had round 1 of deterministic services (if i may call t= hem that) using rfc2212 with RSVP fail because of real or assumed infeasibi= lity to scale on multiple dimensions. IMHO it actually failed also because = of assumed infeasibility becasue there was no good analysis and documentati= on of what actually could be made to work (badmouthing...). >From this experience i can only recommend to make sure that we do understan= d what and how something is feasible to be implemented a what scale and spe= ed.=20 For example, i am not aware to have seen general purpose EDF hardware at sc= ale. But i am very interested for any pointers. On the research side, this = one is the oldest one for PIFO i know: https://eur01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fweb.mit.= edu%2Fpifo%2Fpifo-sigcomm.pdf&data=3D04%7C01%7Cyaakov_s%40rad.com%7Ccb4= 381151b5f494cc2ce08d8e2505295%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1%7C0%7C= 637508181910510455%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu= MzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DLCK5i%2BmLewtZdjJIK7R= lDPFqIRONvgefZxaX1f91ckw%3D&reserved=3D0 [ i am of course mentioning PIFO because by using deadline as the PIFO rank= , one has a simple approach to implement EDF]. In this work, if i remember my analysis correctly, the scale still depends = on the number of flows and the ability to identify packets to flows (need t= o read it again though). This _may_ be acceptable in specific use-cases but should IMHO be well unde= rstood and documented, especially when the view from the outside is that by= using e.g.: SR packet headers it is "looking" as if there is really no per= -flow scaling aspect to the hardware requiremens. After all, the idea of so= urce routing with SR and having the state in the packet is to eliminate the= need to have) and scale it in the router. Cheers Toerless > >> Sorry that's a typo. I mean PIFO (although we do have a paper under re= view using the name PIPO). Yes I agree those are just abstract primitives. = The actual implementation, if customized to a particular algorithm, would b= e simpler. >=20 > Y(J)S >=20 > From: Haoyu Song=20 > > > Sent: 05/03/2021 22:46 > To: Yaakov Stein >;=20 > detnet@ietf.org;=20 > spring@ietf.org;=20 > pce@ietf.org > Subject: RE: new draft on segment routing approach to TSN >=20 >=20 > CAUTION: External sender. Do not click links or open attachments unless y= ou know the content is safe. > Hi Yaakov, >=20 > Just got a chance to read your draft. I agree with the comments of the ot= hers that this is a very interesting work. I'll just add a few points. >=20 >=20 > 1. The use of clock time as deadline requires network synchronization,= and accurate measurement of per-link propagation time, which can somehow l= imit the application scope of this work. Alternatively, one can simply budg= et a device latency which require a router/switch to obey. In case the over= all budget is evenly divided by the hops, a single parameter is enough. Of = course, if one wants to customize the budget on each hop (which might be ne= cessary considering the different capability/capacity of each hop), a stack= is still needed. > 2. Mechanism should be provisioned to track where the timing requireme= nt is violated and by how much (e.g., using timestamp or flag). This would = be very useful for troubleshooting. > 3. Recently programmable scheduler research has proposed several primi= tives such as PIPO and PIEO and provided feasible hardware implementations.= The scheme proposed in this draft can easily fit into these primitives. >=20 > Best regards, > Haoyu > From: spring >=20 > On Behalf Of Yaakov Stein > Sent: Tuesday, February 23, 2021 5:14 AM > To: detnet@ietf.org;=20 > spring@ietf.org;=20 > pce@ietf.org > Subject: [spring] new draft on segment routing approach to TSN >=20 > All, >=20 > I would like to call your attention to a new ID=20 > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww. > ietf.org%2Farchive%2Fid%2Fdraft-stein-srtsn-00.txt&data=3D04%7C01%7C > yaakov_s%40rad.com%7Ccb4381151b5f494cc2ce08d8e2505295%7Cf9047108cc2c4e > 4897a343fad1b3bf9d%7C1%7C0%7C637508181910510455%7CUnknown%7CTWFpbGZsb3 > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7 > C1000&sdata=3DKhdBd9Z2XCTVUvweatNWyCgwZDVKssHJF2W%2Fz41Q1uY%3D&r > eserved=3D0 %2F%2Fwww.ietf.org%2Farchive%2Fid%2Fdraft-stein-srtsn-00.txt&data=3D > 04%7C01%7Cyaakov_s%40rad.com%7Ccb4381151b5f494cc2ce08d8e2505295%7Cf904 > 7108cc2c4e4897a343fad1b3bf9d%7C1%7C0%7C637508181910510455%7CUnknown%7C > TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC > I6Mn0%3D%7C1000&sdata=3DKhdBd9Z2XCTVUvweatNWyCgwZDVKssHJF2W%2Fz41Q1u > Y%3D&reserved=3D0> which describes using a stack-based approach=20 > (similar to segment routing) to time sensitive networking. > It furthermore proposes combining segment routing with this approach=20 > to TSN resulting in a unified approach to forwarding and scheduling. >=20 > The draft is information at this point, since it discusses the concepts a= nd does not yet pin down the precise formats. >=20 > Apologies for simultaneously sending to 3 lists, but I am not sure=20 > which WG is the most appropriate for discussions of this topic. >=20 > * DetNet is most relevant since the whole point is to control end-to-= end latency of a time-sensitive flow. > * Spring is also directly relevant due to the use of a stack in the h= eader and the combined approach just mentioned. > * PCE is relevant to the case of a central server jointly computing a= n optimal path and local deadline stack. > I'll let the chairs decide where discussions should be held. >=20 > Y(J)S >=20 > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww. > ietf.org%2Fmailman%2Flistinfo%2Fdetnet&data=3D04%7C01%7Cyaakov_s%40r > ad.com%7Ccb4381151b5f494cc2ce08d8e2505295%7Cf9047108cc2c4e4897a343fad1 > b3bf9d%7C1%7C0%7C637508181910510455%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC > 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sd > ata=3Dv3Dh%2Bq0%2F8eYutLOjnkz0Aa%2BvieQW8CoRhhbv7JfN77c%3D&reserved= =3D > 0 -- --- tte@cs.fau.de From nobody Tue Mar 9 01:17:50 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A32463A1843; Tue, 9 Mar 2021 01:17:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.247 X-Spam-Level: X-Spam-Status: No, score=-2.247 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ABHlE3tPaNXU; Tue, 9 Mar 2021 01:17:43 -0800 (PST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2046.outbound.protection.outlook.com [40.107.20.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6A4D73A1842; Tue, 9 Mar 2021 01:17:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KKkG3Vm4GGtEqeeMuYTq4NXGws8QurRMD35gQHk2dE+bDt70Zso6iHG18tLzt5dsT/MfJESajnrwVMIrE+mljpLwinrVKHWisjWbx69oSjJ4EduvtMSanQDCMBLJCMgAilOt1WgRTqXQHqEI4v2g7+aqwEWv2j5oWpNGXHHS+2Nxa8LYgBBiTEtlsVctiEqse+FdlZRyOrSMNvUHflR1SMOqcqiUL1JjVsLvbD0Mvt/6AAyome1EvIO3N/Erz8xS8Enu0+D7dVRGo4M3KIwSX46/vgFCztpM7mAE1NDqtse2TR2QJYnUTFMGBlhW5yChT3TQx+sAr0WKxJd1eCTsXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xe2IxXA5oMSO/jpxdnXG7nP3KKIJ7WNFHdye5J6PdsU=; b=OhhF/1/k2IFvauRIGfhQAxu0g3pc7AEqBPe7WJ3M4OVX3EunwpCHWZkhLbbzQSjNIavXgg2hcJvcntRqq+wixLdVR3qOwd2mOq+WLNHfsrEBbDQAZzwyzJn8YIqwq06ZvtLVv0ApIUJ8CgCkpnj7GSWH3BRgJjm/eWyo6VcN6QzNZrEku2RUXPvmRkpylq4beuZ+vQ/iYtzxlC3rebVrKlqLatTjY+fhL+sz6Xrqvc4JlMnUW9P5R1T10nsfCpF/JuGWh70nCekaaUx87bw74CjQvJhfoARPzNvKZiri7PyC/Umy6HO9tZBSKxFWnWL1/T7fyae5z8Z0dMMR1SLo/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xe2IxXA5oMSO/jpxdnXG7nP3KKIJ7WNFHdye5J6PdsU=; b=HMXf6osxXcpbbhrdZre3YJ4vvSSR/cc0gayvk5cFwHm+tU2CbvH6AnpZHRLuXqC9PuJetKFDR0CISQ/HlGBbYoyGpDmVbyJNxSfNx6MOJH+Xn08rTgagrTfIEcbeVaDgHXc7R/spPVJ2bNN0uQ74trYfuBMX9nBFDEgdWF42Hdo= Received: from HE1PR0702MB3609.eurprd07.prod.outlook.com (2603:10a6:7:83::33) by HE1PR0701MB2409.eurprd07.prod.outlook.com (2603:10a6:3:6e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.11; Tue, 9 Mar 2021 09:17:26 +0000 Received: from HE1PR0702MB3609.eurprd07.prod.outlook.com ([fe80::b1b9:13ce:fac6:586b]) by HE1PR0702MB3609.eurprd07.prod.outlook.com ([fe80::b1b9:13ce:fac6:586b%4]) with mapi id 15.20.3933.025; Tue, 9 Mar 2021 09:17:26 +0000 From: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= To: "Liubingyang (Bryan)" , Yaakov Stein , Haoyu Song , detnet , spring , pce Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpAAA9T+5AAAZ19cAAB5aG+ACccnpA= Date: Tue, 9 Mar 2021 09:17:25 +0000 Message-ID: References: <9b01579697ba42cf90b0bd5861730fc3@huawei.com>, F09AC6CE-AECC-4EC0-B0FA-34EB7844EDE2 In-Reply-To: F09AC6CE-AECC-4EC0-B0FA-34EB7844EDE2 Accept-Language: hu-HU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [185.29.82.144] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3d905715-edfb-4aee-3899-08d8e2dc27d7 x-ms-traffictypediagnostic: HE1PR0701MB2409: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: fppyL0mzVlPHCNi7dP+0ftIGhJ7KwVLOw+u9H1HWv2/amy2g9Xs75A0TRDKMRlP+vWS3jaGdJlaRmCsdCTodlRgjNkKHF3yuNRn9x8qMKlCz3ASQThiyFoN93mPxXd68wFQAdmzm+Hi8psVc0l/SJe4sZzbjQqAStIIoQGMJVR87Vjx4Unmh9LZD2th5P9zeiy/MfTmKg1NvJ5b9HLNtE4cY11CaateUfbsLPs5pgJ0BsBs0yNC/4kecHcDUnDv9Qg/B71I7ew0g2dUfEYsWuz4LaPnNwOWtx2zvuOnUituWVKjnPna3P9jua0XZI0LmDh/Svd3DannC04LwIpWRrYluRKLNTa9Qj5rinyQJ4NzmLmhzXxPWhm6xfn89v0nF/kGYGPC9yXdSwyQ3RN0DhYR200+phRdLS5OHDVDE6KKWB+zCreR5SgdWyKTiB+Ep3N5E8TxENW3Lio/onIoSDvyxnQJanS54noT0GgkW32ryiZynb/B8HEzthT8xy/BYSHXEpPpjbsOUDM8Zap9kahf4USgcjqGlcpTUn24ZwzNxiQYNY9Fzh4zp5/0aYLv7nCa/+TBgLg0yEmiizVh/JxWExUnjyCYVWvFNXdTuGH0= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0702MB3609.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(366004)(396003)(39850400004)(66476007)(66946007)(76116006)(86362001)(83380400001)(66446008)(55016002)(508600001)(966005)(66574015)(52536014)(110136005)(71200400001)(8936002)(9686003)(7696005)(26005)(186003)(53546011)(166002)(6506007)(64756008)(8676002)(2906002)(5660300002)(66556008)(9326002)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: uUv2A7D5REaT/6/cVlN7TepTMzldO4afgkvorPNfLloLG+ZLU3algrQ7rmHM6qLR9NhaJ/9eEYIH2BELL8KX8rkeEv751qBEcladsTapm+LtpSn2OM71LSD9fH9mCpqMEKftqkiLL1XoNOxmMZvVMVrLbfozgwRuQCeQ1Igc7cGIF/31mfqpH7eCTolYQLvu47pB48ohfA4TOsCwEkZ3cxVZ3eBLh2v1rtRRFuAH2Xfytju6fQcsRM9xl9A/F2ofqpkhG4kpyPx3bKpLIcMQnUJt8KHLcaIzB68fvMPsJVZy7Nwmves4+GEfNwCAOHleO5DmXamdkSle26D+mjNVeZAvF1s6l+zxvHb2wNN7/HcdSfWvgmdN4ypetb3KVAJvtTgDAygUpsYXHjeXsTrHOcglNuMyAoSv/4X+bkdEE7MEw0lG3ofW0ZK1C9tqpQvxfGrf471iUMpsd14YUAhSqot2TREzIfLXAjkhP20j/mpU39PqehSWsukPLnqtZ+hG94eD+zAZvmLjwNxKRcoi4CA9NFTeJIva4ofy5b6DZ05UAhlJdVopqQz/dloQ2B++Vfzho6TGksHbAzIPD5ddYnuUqKn+ShBe2OcCoy/2antwVLAfNNx/sDeLhH90lXsnM5SHgJ5Xuu1Ux7SQbJQqqBLPvIeuxFdLtVp+UJ8dgDvlJK0y7L8GM+k+ZqdYhS0mq6Ga5bylbEwex0O7dgxvx3jBj+oVZoxut7DfW+oq0LA= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_HE1PR0702MB36092FA11B951E91CA57898CAC929HE1PR0702MB3609_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: HE1PR0702MB3609.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3d905715-edfb-4aee-3899-08d8e2dc27d7 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Mar 2021 09:17:26.0074 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GpRQ7vxXI0l0QLcWDxE1ToT7lM35AOh/ttuoXTff7YF+e+W+RCGSM4a3i1smgY7Iole5NvbKKUdnmgAdvvU7CVVLCETTatfEQyhUtLecHRs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0701MB2409 Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 09:17:46 -0000 --_000_HE1PR0702MB36092FA11B951E91CA57898CAC929HE1PR0702MB3609_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Bryan, Sure, but there is no free lunch. :--)) You have added the problem of calculation of the budget. Knowing your exact egress time in advance is not an easy task. There will be an inaccuracy in budget calculations. Cheers Bala'zs From: detnet On Behalf Of Liubingyang (Bryan) Sent: Monday, March 8, 2021 3:31 PM To: Yaakov Stein ; Haoyu Song ;= detnet ; spring ; pce Subject: Re: [Detnet] new draft on segment routing approach to TSN Some advantages. If you use budget instead of deadline, you don't need accurate time sync. If you use budget, the time budget encoded in packet require fewer bits tha= n using absolute time. The tightness is related to whether and how many flows can be admitted. It = is related to network resource utilization and revenue. If e2e deadline is 1ms, then time sync with 1ms accuracy will not work well= . Using budget helps resolve this problem. Just my two cents. Bryan From:Yaakov Stein > To:Liubingyang (Bryan) >;Haoyu Song >;= detnet >;spring >;pce > Date:2021-03-08 21:38:20 Subject:RE: new draft on segment routing approach to TSN Bryan The local deadlines are specified in absolute time - not deltas. So there is no need to add the time saved - it is already there! As I previously replied, the accuracy of the time sync depends on the tight= ness of the budget. It has nothing to do with the encoding. Y(J)S From: Liubingyang (Bryan) > Sent: 08/03/2021 14:55 To: Yaakov Stein >; Haoyu Song >; detnet@ietf.org<= mailto:detnet@ietf.org>; spring@ietf.org; pce@ietf.= org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Great work. One suggestion. If we divide the end-to-end delay budget to per-hop budgets, and carry them= in SR header, we can mimic using the end-to-end absolute deadline. One thi= ng we need to do is, when the actual time used in a hop is less than the bu= dget of this hop, you can add the remaining budget to the budget of next ho= p. In this case, you can save a lot of bits used for time, and you don't ha= ve to rely on accurate time sync. Bryan (Bingyang Liu) From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Monday, March 8, 2021 1:37 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We'll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song = > Sent: 08/03/2021 04:41 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_HE1PR0702MB36092FA11B951E91CA57898CAC929HE1PR0702MB3609_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Bryan,

Sure, but there is no free lunch. :--))
You have added the problem of calculation of the bud= get.

Knowing your exact egress time in advance is not an = easy task.

There will be an inaccuracy in budget calculations.<= o:p>

Cheers

Bala’zs

From: detnet <detnet-bounces@ietf.org> = On Behalf Of Liubingyang (Bryan)
Sent: Monday, March 8, 2021 3:31 PM
To: Yaakov Stein <yaakov_s@rad.com>; Haoyu Song <haoyu.song= @futurewei.com>; detnet <detnet@ietf.org>; spring <spring@ietf.= org>; pce <pce@ietf.org>
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Some advantages.

If you use budget instead of deadline, you don't nee= d accurate time sync.

If you use budget, the time budget encoded in packet= require fewer bits than using absolute time.

The tightness is related to whether and how many flo= ws can be admitted. It is related to network resource utilization and reven= ue.

If e2e deadline is 1ms, then time sync with 1ms accu= racy will not work well. Using budget helps resolve this problem.

Just my two cents.

Bryan

From:Yaakov St= ein <yaakov_s@rad.com>

To:Liubingyang= (Bryan) <liubingyang@huawei.c= om>;Haoyu Song <haoyu= .song@futurewei.com>;detnet <d= etnet@ietf.org>;spring <spring@ietf.org>;pce <pce@ietf.org>

Date:2021-03-0= 8 21:38:20

Subject:RE: ne= w draft on segment routing approach to TSN

Bryan=

= ;

The l= ocal deadlines are specified in absolute time – not deltas.

So th= ere is no need to add the time saved – it is already there!

= ;

As I = previously replied, the accuracy of the time sync depends on the tightness = of the budget.

It ha= s nothing to do with the encoding.

= ;

Y(J)S=

From: Liubingyang (Bryan) <liubingyang@huawei.com>
Sent: 08/03/2021 14:55
To: Yaakov Stein <yaakov_s@ra= d.com>; Haoyu Song <h= aoyu.song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Ya= akov,

= ;

Great= work. One suggestion.

If we= divide the end-to-end delay budget to per-hop budgets, and carry them in S= R header, we can mimic using the end-to-end absolute deadline. One thing we= need to do is, when the actual time used in a hop is less than the budget of this hop, you can add the remaini= ng budget to the budget of next hop. In this case, you can save a lot of bi= ts used for time, and you don’t have to rely on accurate time sync.

= ;

Bryan= (Bingyang Liu)

= ;

From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein
Sent: Monday, March 8, 2021 1:37 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Haoyu

I think we are in agreement.

I do not see the need for explicitly handling the ca= se of a packet missing a LOCAL deadline,

since the following switches will already handle thi= s case optimally (and may still meet the overall budget).

Counting packets that miss their delay budget is ind= eed important,

and a counter could be configured in the egress rout= er for this.

We’ll need to define this when we get to the p= rotocol specification.

It would be advantageous to put a threshold on the f= ailure rate

and feed this back to the path/stack optimizer.=

Y(J)S

From: Haoyu Song <haoyu.song@futurewei.com>
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Some feedback inline.

Best regards,

Haoyu

From: Yaakov Stein <yaakov_s@rad.com>
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Haoyu,

I’ll address your points:

> The use of clock time as deadline requires netw= ork synchronization

That is a basic assumption of TSN networks (IMO the = defining assumption).

However, the sync needn’t be highly accurate &= #8211; it depends on the tightness of the delay budgets.

>> Yes, I understand it. What I mean is that t= he method mentioned in the draft seems to be also applicable to other types= of networks. For example, we can envision some time-critical traffic in DC= N or WAN. If a protocol would be developed, can it also serve the other networks? If so, it would be better.

> accurate measurement of per-link propagation ti= me

Yes, I am assuming that once an for all someone does= a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measurement of the links, a= nd these are stored in some database.

Once again, the required accuracy depends on the del= ay budgets.

> which can somehow limit the application scope o= f this work

If the delay budget only above the physically minima= l delay by say less than 100 microseconds,

I agree that the previous two issues MUST be carried= out. But in such cases there is no alternative.

If the delay budget is much higher than that, then o= ne could use an RSVP-like mechanism,

sending a packet (or several packets) from source to= destination collecting a stack of timestamps,

and then using that stack for the following packets.=

> Mechanism should be provisioned to track where = the timing requirement is violated and by how much

I’ll leave the OAM for later. However there ar= e already many high accuracy performance measurement techniques and protoco= ls for this.

>> For this I mean something recorded in the s= ame packet with the deadlines. If it misses the deadline, the receiver may = need to know where it’s violated. Other independent methods are possi= ble, but it’s better to consider if it can be integrated in the current proposal.

> Recently programmable scheduler research has pr= oposed several primitives

Yes, I tried to stress that this ID is not limited t= o EDF (although sometimes that is a good strategy).

One can even reproduce Qbv behavior using a stack of= deadlines (although why would one wish to do so?).

> such as PIPO and PIEO

I’ve heard of PIFO (Push In First out) but not= PIPO. Is this a typo or something new?

I agree that there are mechanisms that are optimized= for hardware, but I have come up with a very nice hardware implementation = for PEDF

and prefer to find hardware implementations for opti= mal schedulers, rather than to determine schedulers based on optimal hardwa= re.

>> Sorry that’s a typo. I mean PIFO (alt= hough we do have a paper under review using the name PIPO). Yes I agree tho= se are just abstract primitives. The actual implementation, if customized t= o a particular algorithm, would be simpler.

Y(J)S

From: Haoyu Song <haoyu.song@futurewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not clic= k links or open attachments unless you know the content is safe.

Hi Yaakov,

Just got a chance to read your draft. I agree with t= he comments of the others that this is a very interesting work. I’ll = just add a few points.

The use of clock = time as deadline requires network synchronization, and accurate measurement= of per-link propagation time, which can somehow limit the application scop= e of this work. Alternatively, one can simply budget a device latency which require a router/switch to obey. In c= ase the overall budget is evenly divided by the hops, a single parameter is= enough. Of course, if one wants to customize the budget on each hop (which= might be necessary considering the different capability/capacity of each hop), a stack is still needed. <= o:p>
Me= chanism should be provisioned to track where the timing requirement is viol= ated and by how much (e.g., using timestamp or flag). This would be very us= eful for troubleshooting.
Recently programmable scheduler research has propos= ed several primitives such as PIPO and PIEO and provided feasible hardware = implementations. The scheme proposed in this draft can easily fit into thes= e primitives.

Best regards,

Haoyu

From: spring <spring-bounces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your attention to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (simila= r to segment routing) to time sensitive networking.

It furthermore proposes combining segment routing wi= th this approach to TSN

resulting in a unified approach to forwarding and sc= heduling.

The draft is information at this point, since it dis= cusses the concepts and does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,

but I am not sure which WG is the most appropriate f= or discussions of this topic.

DetNet is most re= levant since the whole point is to control end-to-end latency of a time-sen= sitive flow.
Spring is also directly relevant due to the use of a stack in th= e header and the combined approach just mentioned.
PCE is relevant to the cas= e of a central server jointly computing an optimal path and local deadline = stack.

I’ll let the chairs decide where discussions s= hould be held.

Y(J)S

--_000_HE1PR0702MB36092FA11B951E91CA57898CAC929HE1PR0702MB3609_-- From nobody Tue Mar 9 02:14:11 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C94323A1930; Tue, 9 Mar 2021 02:14:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.798 X-Spam-Level: X-Spam-Status: No, score=-1.798 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bcnmQJeMbVgJ; Tue, 9 Mar 2021 02:14:06 -0800 (PST) Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1282F3A192F; Tue, 9 Mar 2021 02:14:06 -0800 (PST) Received: from fraeml711-chm.china.huawei.com (unknown [172.18.147.206]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DvrSk2Xpzz67xN4; Tue, 9 Mar 2021 18:05:58 +0800 (CST) Received: from dggeme753-chm.china.huawei.com (10.3.19.99) by fraeml711-chm.china.huawei.com (10.206.15.60) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2106.2; Tue, 9 Mar 2021 11:13:57 +0100 Received: from dggeme751-chm.china.huawei.com (10.3.19.97) by dggeme753-chm.china.huawei.com (10.3.19.99) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Tue, 9 Mar 2021 18:13:55 +0800 Received: from dggeme751-chm.china.huawei.com ([169.254.210.30]) by dggeme751-chm.china.huawei.com ([169.254.210.30]) with mapi id 15.01.2106.013; Tue, 9 Mar 2021 18:13:55 +0800 From: "Liubingyang (Bryan)" To: =?iso-8859-1?Q?Bal=E1zs_Varga_A?= , "Yaakov Stein" , Haoyu Song , detnet , spring , pce Thread-Topic: new draft on segment routing approach to TSN Thread-Index: AdcJ5AjgmuXpLt94R1Stsoh/vUDwUAIGpPHgAEKRNTAALm7rkAAGaIpAAA9T+5AAAZ19cAAB5aG+ACccnpAAAcdGAA== Date: Tue, 9 Mar 2021 10:13:55 +0000 Message-ID: References: <9b01579697ba42cf90b0bd5861730fc3@huawei.com>, F09AC6CE-AECC-4EC0-B0FA-34EB7844EDE2 In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.108.234.163] Content-Type: multipart/alternative; boundary="_000_b3746c797d8f4371919100f478390164huaweicom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: Subject: Re: [Detnet] new draft on segment routing approach to TSN X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 10:14:09 -0000 --_000_b3746c797d8f4371919100f478390164huaweicom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable You are absolutely right : ) I see the remaining budget calculation as the difficulty in device implemen= tation. But I see the bit length as protocol overhead, and time sync as deployment = difficulty. Actually all possibilities can be discussed, and when used in different sce= narios, different profiles (combinations of modules) may be used. Best Bryan From: Bal=E1zs Varga A [mailto:balazs.a.varga@ericsson.com] Sent: Tuesday, March 9, 2021 5:17 PM To: Liubingyang (Bryan) ; Yaakov Stein ; Haoyu Song ; detnet ; sp= ring ; pce Subject: RE: new draft on segment routing approach to TSN Hi Bryan, Sure, but there is no free lunch. :--)) You have added the problem of calculation of the budget. Knowing your exact egress time in advance is not an easy task. There will be an inaccuracy in budget calculations. Cheers Bala'zs From: detnet > On B= ehalf Of Liubingyang (Bryan) Sent: Monday, March 8, 2021 3:31 PM To: Yaakov Stein >; Haoyu Song >; detnet >; spring >; pce > Subject: Re: [Detnet] new draft on segment routing approach to TSN Some advantages. If you use budget instead of deadline, you don't need accurate time sync. If you use budget, the time budget encoded in packet require fewer bits tha= n using absolute time. The tightness is related to whether and how many flows can be admitted. It = is related to network resource utilization and revenue. If e2e deadline is 1ms, then time sync with 1ms accuracy will not work well= . Using budget helps resolve this problem. Just my two cents. Bryan From:Yaakov Stein > To:Liubingyang (Bryan) >;Haoyu Song >;= detnet >;spring >;pce > Date:2021-03-08 21:38:20 Subject:RE: new draft on segment routing approach to TSN Bryan The local deadlines are specified in absolute time - not deltas. So there is no need to add the time saved - it is already there! As I previously replied, the accuracy of the time sync depends on the tight= ness of the budget. It has nothing to do with the encoding. Y(J)S From: Liubingyang (Bryan) > Sent: 08/03/2021 14:55 To: Yaakov Stein >; Haoyu Song >; detnet@ietf.org<= mailto:detnet@ietf.org>; spring@ietf.org; pce@ietf.= org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Great work. One suggestion. If we divide the end-to-end delay budget to per-hop budgets, and carry them= in SR header, we can mimic using the end-to-end absolute deadline. One thi= ng we need to do is, when the actual time used in a hop is less than the bu= dget of this hop, you can add the remaining budget to the budget of next ho= p. In this case, you can save a lot of bits used for time, and you don't ha= ve to rely on accurate time sync. Bryan (Bingyang Liu) From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Monday, March 8, 2021 1:37 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: Re: [Detnet] new draft on segment routing approach to TSN Haoyu I think we are in agreement. I do not see the need for explicitly handling the case of a packet missing = a LOCAL deadline, since the following switches will already handle this case optimally (and m= ay still meet the overall budget). Counting packets that miss their delay budget is indeed important, and a counter could be configured in the egress router for this. We'll need to define this when we get to the protocol specification. It would be advantageous to put a threshold on the failure rate and feed this back to the path/stack optimizer. Y(J)S From: Haoyu Song = > Sent: 08/03/2021 04:41 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN Hi Yaakov, Some feedback inline. Best regards, Haoyu From: Yaakov Stein > Sent: Saturday, March 6, 2021 8:36 PM To: Haoyu Song >;= detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: RE: new draft on segment routing approach to TSN Haoyu, I'll address your points: > The use of clock time as deadline requires network synchronization That is a basic assumption of TSN networks (IMO the defining assumption). However, the sync needn't be highly accurate - it depends on the tightness = of the delay budgets. >> Yes, I understand it. What I mean is that the method mentioned in the dr= aft seems to be also applicable to other types of networks. For example, we= can envision some time-critical traffic in DCN or WAN. If a protocol would= be developed, can it also serve the other networks? If so, it would be bet= ter. > accurate measurement of per-link propagation time Yes, I am assuming that once an for all someone does a TDR or at least OWAM= P/TWAMP/Y.1731-1wD delay measurement of the links, and these are stored in = some database. Once again, the required accuracy depends on the delay budgets. > which can somehow limit the application scope of this work If the delay budget only above the physically minimal delay by say less tha= n 100 microseconds, I agree that the previous two issues MUST be carried out. But in such cases= there is no alternative. If the delay budget is much higher than that, then one could use an RSVP-li= ke mechanism, sending a packet (or several packets) from source to destination collecting= a stack of timestamps, and then using that stack for the following packets. > Mechanism should be provisioned to track where the timing requirement is = violated and by how much I'll leave the OAM for later. However there are already many high accuracy = performance measurement techniques and protocols for this. >> For this I mean something recorded in the same packet with the deadlines= . If it misses the deadline, the receiver may need to know where it's viola= ted. Other independent methods are possible, but it's better to consider if= it can be integrated in the current proposal. > Recently programmable scheduler research has proposed several primitives Yes, I tried to stress that this ID is not limited to EDF (although sometim= es that is a good strategy). One can even reproduce Qbv behavior using a stack of deadlines (although wh= y would one wish to do so?). > such as PIPO and PIEO I've heard of PIFO (Push In First out) but not PIPO. Is this a typo or some= thing new? I agree that there are mechanisms that are optimized for hardware, but I ha= ve come up with a very nice hardware implementation for PEDF and prefer to find hardware implementations for optimal schedulers, rather = than to determine schedulers based on optimal hardware. >> Sorry that's a typo. I mean PIFO (although we do have a paper under revi= ew using the name PIPO). Yes I agree those are just abstract primitives. Th= e actual implementation, if customized to a particular algorithm, would be = simpler. Y(J)S From: Haoyu Song = > Sent: 05/03/2021 22:46 To: Yaakov Stein >; detnet@ietf.o= rg; spring@ietf.org; pce@ie= tf.org Subject: RE: new draft on segment routing approach to TSN CAUTION: External sender. Do not click links or open attachments unless you= know the content is safe. Hi Yaakov, Just got a chance to read your draft. I agree with the comments of the othe= rs that this is a very interesting work. I'll just add a few points. 1. The use of clock time as deadline requires network synchronization, a= nd accurate measurement of per-link propagation time, which can somehow lim= it the application scope of this work. Alternatively, one can simply budget= a device latency which require a router/switch to obey. In case the overal= l budget is evenly divided by the hops, a single parameter is enough. Of co= urse, if one wants to customize the budget on each hop (which might be nece= ssary considering the different capability/capacity of each hop), a stack i= s still needed. 2. Mechanism should be provisioned to track where the timing requirement= is violated and by how much (e.g., using timestamp or flag). This would be= very useful for troubleshooting. 3. Recently programmable scheduler research has proposed several primiti= ves such as PIPO and PIEO and provided feasible hardware implementations. T= he scheme proposed in this draft can easily fit into these primitives. Best regards, Haoyu From: spring > On B= ehalf Of Yaakov Stein Sent: Tuesday, February 23, 2021 5:14 AM To: detnet@ietf.org; spring@ietf.org; pce@ietf.org Subject: [spring] new draft on segment routing approach to TSN All, I would like to call your attention to a new ID https://www.ietf.org/archiv= e/id/draft-stein-srtsn-00.txt which describes using a stack-based approach (similar to segment routing) t= o time sensitive networking. It furthermore proposes combining segment routing with this approach to TSN resulting in a unified approach to forwarding and scheduling. The draft is information at this point, since it discusses the concepts and= does not yet pin down the precise formats. Apologies for simultaneously sending to 3 lists, but I am not sure which WG is the most appropriate for discussions of this = topic. * DetNet is most relevant since the whole point is to control end-to-en= d latency of a time-sensitive flow. * Spring is also directly relevant due to the use of a stack in the hea= der and the combined approach just mentioned. * PCE is relevant to the case of a central server jointly computing an = optimal path and local deadline stack. I'll let the chairs decide where discussions should be held. Y(J)S --_000_b3746c797d8f4371919100f478390164huaweicom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

You are absolutely right : )

I see the remaining budget calculation as the difficulty in devic= e implementation.

But I see the bit length as protocol overhead, and time sync as d= eployment difficulty.

Actually all possibilities can be discussed, and when used in dif= ferent scenarios, different profiles (combinations of modules) may be used.

Best

Bryan

From: Bal=E1zs Varga A [mailto:balazs.a.varga@ericsson.com]
Sent: Tuesday, March 9, 2021 5:17 PM
To: Liubingyang (Bryan) <liubingyang@huawei.com>; Yaakov Stein= <yaakov_s@rad.com>; Haoyu Song <haoyu.song@futurewei.com>; det= net <detnet@ietf.org>; spring <spring@ietf.org>; pce <pce@ie= tf.org>
Subject: RE: new draft on segment routing approach to TSN=

Hi Bryan,

Sure, but there is no free lunc= h. :--))

You have added the problem of c= alculation of the budget.

Knowing your exact egress time = in advance is not an easy task.

There will be an inaccuracy in = budget calculations.

Cheers

Bala’zs=

From: detnet <detnet-bo= unces@ietf.org> On Behalf Of Liubingyang (Bryan)
Sent: Monday, March 8, 2021 3:31 PM
To: Yaakov Stein <yaakov_s@ra= d.com>; Haoyu Song <h= aoyu.song@futurewei.com>; detnet <detnet@ietf.org>; spring <= spring@ietf.org>; pce <pce@ietf.org>
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Some advantages.

If you use budget instead of de= adline, you don't need accurate time sync.

If you use budget, the time bud= get encoded in packet require fewer bits than using absolute time.

The tightness is related to whe= ther and how many flows can be admitted. It is related to network resource = utilization and revenue.

If e2e deadline is 1ms, then ti= me sync with 1ms accuracy will not work well. Using budget helps resolve th= is problem.

Just my two cents. <= /span>

=
Bryan

From:Yaakov Stein <yaakov_s@rad.com>

To:Liubingyang (Bryan) <liubingyang@huawei.com>;Haoyu Song <= haoyu.song@futurewei.com>= ;;detnet <detnet@ietf.org>;spring <= spring@ietf.org>;pce <pce@ietf.org>

Date:2021-03-08 21:38:20

Subject:RE: new draft on segment routing= approach to TSN

Bryan

The local deadlines are specified in absolute time – not de= ltas.

So there is no need to add the time saved – it is already t= here!

As I previously replied, the accuracy of the time sync depends on= the tightness of the budget.=

It has nothing to do with the encoding.

Y(J)S

From: Liubingyang (Bryan) <liubingyang@huawei.com>
Sent: 08/03/2021 14:55
To: Yaakov Stein <yaakov_s@ra= d.com>; Haoyu Song <h= aoyu.song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Great work. One suggestion.

If we divide the end-to-end delay budget to per-hop budgets, and = carry them in SR header, we can mimic using the end-to-end absolute deadlin= e. One thing we need to do is, when the actual time used in a hop is less than the budget of this hop, you can add= the remaining budget to the budget of next hop. In this case, you can save= a lot of bits used for time, and you don’t have to rely on accurate = time sync.

Bryan (Bingyang Liu)

From: detnet [mailto:detne= t-bounces@ietf.org] On Behalf Of Yaakov Stein
Sent: Monday, March 8, 2021 1:37 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Haoyu

I think we are in agreement.

I do not see the need for expli= citly handling the case of a packet missing a LOCAL deadline,

since the following switches wi= ll already handle this case optimally (and may still meet the overall budge= t).

Counting packets that miss thei= r delay budget is indeed important,

and a counter could be configur= ed in the egress router for this.

We’ll need to define this= when we get to the protocol specification.

It would be advantageous to put= a threshold on the failure rate

and feed this back to the path/= stack optimizer.

Y(J)S

From: Haoyu Song <haoy= u.song@futurewei.com>
Sent: 08/03/2021 04:41
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Hi Yaakov,

Some feedback inline.

Best regards,=

Haoyu

From: Yaakov Stein <yaakov_s@r= ad.com>
Sent: Saturday, March 6, 2021 8:36 PM
To: Haoyu Song <haoyu= .song@futurewei.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

Haoyu,

I’ll address your points:=

> The use of clock time as d= eadline requires network synchronization

That is a basic assumption of T= SN networks (IMO the defining assumption).

However, the sync needn’t= be highly accurate – it depends on the tightness of the delay budget= s.

>> Yes, I understand it. = What I mean is that the method mentioned in the draft seems to be also appl= icable to other types of networks. For example, we can envision some time-c= ritical traffic in DCN or WAN. If a protocol would be developed, can it also serve the other networks? If so, it would = be better.

> accurate measurement of pe= r-link propagation time

Yes, I am assuming that once an= for all someone does a TDR or at least OWAMP/TWAMP/Y.1731-1wD delay measur= ement of the links, and these are stored in some database.

Once again, the required accura= cy depends on the delay budgets.

> which can somehow limit th= e application scope of this work

If the delay budget only above = the physically minimal delay by say less than 100 microseconds,

I agree that the previous two i= ssues MUST be carried out. But in such cases there is no alternative.<= /o:p>

If the delay budget is much hig= her than that, then one could use an RSVP-like mechanism,=

sending a packet (or several pa= ckets) from source to destination collecting a stack of timestamps,

and then using that stack for t= he following packets.

> Mechanism should be provis= ioned to track where the timing requirement is violated and by how much

I’ll leave the OAM for la= ter. However there are already many high accuracy performance measurement t= echniques and protocols for this.

>> For this I mean someth= ing recorded in the same packet with the deadlines. If it misses the deadli= ne, the receiver may need to know where it’s violated. Other independ= ent methods are possible, but it’s better to consider if it can be integrated in the current proposal.

> Recently programmable sche= duler research has proposed several primitives

Yes, I tried to stress that thi= s ID is not limited to EDF (although sometimes that is a good strategy).

One can even reproduce Qbv beha= vior using a stack of deadlines (although why would one wish to do so?).

> such as PIPO and PIEO=

I’ve heard of PIFO (Push = In First out) but not PIPO. Is this a typo or something new?

I agree that there are mechanis= ms that are optimized for hardware, but I have come up with a very nice har= dware implementation for PEDF

and prefer to find hardware imp= lementations for optimal schedulers, rather than to determine schedulers ba= sed on optimal hardware.

>> Sorry that’s a t= ypo. I mean PIFO (although we do have a paper under review using the name P= IPO). Yes I agree those are just abstract primitives. The actual implementa= tion, if customized to a particular algorithm, would be simpler.

Y(J)S

From: Haoyu Song <haoy= u.song@futurewei.com>
Sent: 05/03/2021 22:46
To: Yaakov Stein <yaakov_s@ra= d.com>; detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN=

CAUTION: External sender. Do not click links or open attachments unless you know the content is safe.

Hi Yaakov= ,

Just got a chance to read your = draft. I agree with the comments of the others that this is a very interest= ing work. I’ll just add a few points.

The use of clock time as deadline requires network synchronization, and= accurate measurement of per-link propagation time, which can somehow limit= the application scope of this work. Alternatively, one can simply budget a device latency which require a router/switch to ob= ey. In case the overall budget is evenly divided by the hops, a single para= meter is enough. Of course, if one wants to customize the budget on each ho= p (which might be necessary considering the different capability/capacity of each hop), a stack is still needed. <= o:p>
Mechanism should be provisioned to track where th= e timing requirement is violated and by how much (e.g., using timestamp or = flag). This would be very useful for troubleshooting.
Recently programmable scheduler research has proposed several primitiv= es such as PIPO and PIEO and provided feasible hardware implementations. Th= e scheme proposed in this draft can easily fit into these primitives.

Best regards,=

Haoyu

From: spring <spring-bo= unces@ietf.org> On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 5:14 AM
To: detnet@ietf.org; spring@ietf.org; pce@ietf.org
Subject: [spring] new draft on segment routing approach to TSN<= /o:p>

All,

I would like to call your atten= tion to a new ID https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt

which describes using a stack-b= ased approach (similar to segment routing) to time sensitive networking.

It furthermore proposes combini= ng segment routing with this approach to TSN

resulting in a unified approach= to forwarding and scheduling.

The draft is information at thi= s point, since it discusses the concepts and does not yet pin down the prec= ise formats.

Apologies for simultaneously se= nding to 3 lists,

but I am not sure which WG is t= he most appropriate for discussions of this topic.

DetNet is most relevant since the whole point is to control end-to-end = latency of a time-sensitive flow.
Spring is also = directly relevant due to the use of a stack in the header and the combined = approach just mentioned.
PCE is relevant to the c= ase of a central server jointly computing an optimal path and local deadlin= e stack.

I’ll let the chairs decid= e where discussions should be held.

Y(J)S

--_000_b3746c797d8f4371919100f478390164huaweicom_-- From nobody Tue Mar 9 06:12:53 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DFF2A3A0D4D for ; Tue, 9 Mar 2021 06:12:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=labn.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IDwUoD7_cns6 for ; Tue, 9 Mar 2021 06:12:49 -0800 (PST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2110.outbound.protection.outlook.com [40.107.94.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1F97B3A0D47 for ; Tue, 9 Mar 2021 06:12:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LVxIeRKr3p0+QRwpk7G00qUJHAK251jQUoG6jURwwS7ddkfzLy3/1NaqRpSvNY6faDUXNhbz9iOoOQTDMHEg3F7Fp1y+B2L6dSfWWaq/FS5UY2vZSiDpWrGNq3LJodWEJpte/GyHroqLFKIMQ8RVfmlxmBiclbl0zHSbdL95IPKZaCnlUjHu360B+ELRseun0OEo92uAtfKqfLP6WKIYadZLU1Et0FTaT9h6IJlcQG/HFGPhrK2hdlRVBC8wqKGIX95TDw9qlY7bqEQHyV8I0boFIbRFbMHSqXxThds/QDjrkhFkziuNTL6O+RLQvRjQds/lZUWDx+c/r/VQP3F43w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GAiHo+rFjHe/cHdF7SDu4ggaO+951JNfI8vbzxe8sf8=; b=B/+JHNwrJj6vwvqPZoKaEE5iicNSKdWbSoXJruhxfQTj32rjmIS635vZKlG8yW+SR/ywakLObMayrLMWE3fuLcrzPwOnUHpmRhQUENyJP8D8j9/ZQXgWwyav0orG3Gtws9AOJiZiBEyPYdmq0CbSAsi/kmxwgB9BH6jua35Q/QK5nzwPOkWiQUyAdQWkDqOdqcGuVXqTcPtBk1u9Iby9PfDO69wXi9oe6vmPG57jteQW3DVDvTG+qzvzoEERQJC/I033cePp7rZq0DqFCZkJmkHRPk91v0SuQdBnMJZvMORlM0uFdJe94ymf8lqCXZDX3CG6M/994SZucltoHANVXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=labn.net; dmarc=pass action=none header.from=labn.net; dkim=pass header.d=labn.net; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=labn.onmicrosoft.com; s=selector2-labn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GAiHo+rFjHe/cHdF7SDu4ggaO+951JNfI8vbzxe8sf8=; b=P8vSmOZVedSO7t5girMXhVsZqYPCETItSSe1++v2AJAH5J+hQdBgptnKJuBXgprvOk1YIxANXdt2xY6wr9p4Hww8iX/d4+wFJbUH+fqrY4p/K71J8fqUq/4IZLxMq4x7eGQvHvvNFcjCG2iI4TqsjsAyqpi5vRWViQyXjlI7yB8= Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=labn.net; Received: from BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) by MN2PR14MB3407.namprd14.prod.outlook.com (2603:10b6:208:1aa::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17; Tue, 9 Mar 2021 14:12:46 +0000 Received: from BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b]) by BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b%7]) with mapi id 15.20.3912.027; Tue, 9 Mar 2021 14:12:45 +0000 From: Lou Berger To: DetNet WG References: <69217591-0d15-63a7-1534-69c7a4db6aee@labn.net> Message-ID: <63970a2d-cb3b-f6d4-fb95-f3bffe34ec13@labn.net> Date: Tue, 9 Mar 2021 09:12:44 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 In-Reply-To: <69217591-0d15-63a7-1534-69c7a4db6aee@labn.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [100.15.108.238] X-ClientProxiedBy: BL0PR02CA0119.namprd02.prod.outlook.com (2603:10b6:208:35::24) To BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [127.0.0.1] (100.15.108.238) by BL0PR02CA0119.namprd02.prod.outlook.com (2603:10b6:208:35::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 14:12:45 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4fb85dc0-d1ac-4384-3488-08d8e305697e X-MS-TrafficTypeDiagnostic: MN2PR14MB3407: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lt6BoaCE5m25UaJwHuBCO3E02eZ762k3GDFaV4WWyoshSozRMKKqdQ9JjP9OH62slb+ELGDWXemLidmoa26yGsfAKFRGIXyD9iuvdDJT2DzjMzx5x+8UmQw4N2zygBoSOsWjFzz/pmE2K5sX1axomkGRZWagkhN37gDLfcQcBBPmsTmlBc9GXhDKstMWwEbcJxmB8kj9IM9T4E47alQntBnmfgAQ5HILB3ILrWoF5Q6Gj8LAOz7CzZ0RjNhga5vwJD9b+CS76ycoeRRaqAjMJClQzSlKwDX4jAwnyeLoIK6uM5glcH+61AowlfYyYp3fralr3/I7VK3sWdeAu0x9I8D/7VF5aty9QmWorruXCM6wQgXhvHdl+h3nHi621bfmTOashnMMaApc7bURbk9H3mDxN4yxKHL5dylygjGis+jMt0e7gMf9SODhlBj33g9yhSUdIGez3PknQX39tN2kHkqt8V4x5ZdegcpY7j8/pRCIaXnBpeVlnaEBQVPKLC6FBEq/WfG9ajbz/Toi6uXYrZga7qwzERIbFTaGtgxMl7OUE4UEJlXgWDWbD7f8pS+itAk+oJhRqYuV5+gqfwArxidE6omoJMO6G7k93v6Lhrbu6doWN9xzBAzceYwE7crRI4DQU8qVekEahM/Z5YHOlbNagdMlPdJckSRz/DATN18H3PvEwM4UcDmfofBH6HkeN6VwbuSrQWlB5cLLYXsRlQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR14MB3779.namprd14.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(346002)(52116002)(66476007)(66556008)(6486002)(83380400001)(8936002)(36756003)(2616005)(16526019)(8676002)(86362001)(16576012)(6916009)(53546011)(31696002)(2906002)(508600001)(186003)(966005)(19273905006)(5660300002)(45640500001)(31686004)(956004)(26005)(66946007)(7126003)(7246003)(43740500002)(45980500001)(563064011); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?cFEyRzcvVmVTSVhveE9jVlo1MlBJM2hMZFEvRFBzK2Eyc2NSeFdubklTN3Q5?= =?utf-8?B?TzdVRGZoWUhYU2dLRGMwM2JlM1dYUFplaFY2bFZkaTBQVnRxTlZyc0NyaWtR?= =?utf-8?B?dkdoOFVpb0R3RHFCWmhCMUlSTXJsd1ovMU1mKzV1RHpmR3pXQ0JiZHFHUWFy?= =?utf-8?B?MlFOZjlaNnQyUE95OUJBSlY5VUZVd1h2VWJnc0U5aFVHZ0ZyUjBWOXBLbm92?= =?utf-8?B?RWtxNjZZdTFqRHJHNTdmcmF3bm12bHlXcytzcyt0bStET281Nno5RjhxWHpx?= =?utf-8?B?aFV1SjFtbTQvV2dDYzhwL0czS3JVMXV5bWJOb2Q2THMrTUtBTGlMQkFjWlhj?= =?utf-8?B?Zm1OQzhOUjN1R2hVU1ZBVG5WQlBWOWpLUG55aTl3RWpENXNQMmZhblJ6RVdj?= =?utf-8?B?Qng3aFFvL01CN3B3OTVCSFp0SWM1MFJpV1FtdTFocDFBTUMxWVVWRVgweVF0?= =?utf-8?B?WGYveUJEdkxVTlNhYys5UFpOZE0zVk12MWVjOVI5VTdraVRMcUQ1WDhmbUVk?= =?utf-8?B?WkRQU0UreGpkNW1aN0ZDbjZ3WmF2MUxkTG52T0hlRlBkblU2MlN3VEhPWEww?= =?utf-8?B?a3FYbng4cE13RHdRSjdKZ29NQmNSaVhrNHJFbkhIK2d6Z0MvQ24vODU1VkVT?= =?utf-8?B?bWNkcTFPN3RLMlpPemFtamQ1SGZJSGFVek9IUGlvV0V2am9zTmJuOXhJeGh3?= =?utf-8?B?T0tQUGV2NndMcEZteDNkK2FtNUJhZVJZOW80a08zTVRCTjRlb0VpWGFZYUpT?= =?utf-8?B?eDcrbG9uc2g4QzlVMGwyTnRrbWE4YkVlYTF5Y2RlbEh4bktSUnp6NkdBYXY1?= =?utf-8?B?UCtqQkxjcmRCb1pVVFVtN2x3eEh1VXRsK01CQitUdm9lVk5uM28xM0YvN0J1?= =?utf-8?B?eDM5UVRrYlhVMlRkRUtoWmlFV2wreEs1a1JUbnJaRjMreHFTUFN0RU40enVw?= =?utf-8?B?K1lIYkpoQXBMN21DWnB0QWNqa1REWWFVMEJDU0tKT2tYT3p4N1NwYys3eVdN?= =?utf-8?B?MzRkT0t6dHB3ZXhCVmRzbUVOTVBJZ3VYQlh2VE9FM3FGcDBVeEgvKzU2T0FJ?= =?utf-8?B?VFBLWWIyT280bnlxNW5YRUhsY1JyM2hjbGJvWksyakV0VS9xRldTTURGR2Rh?= =?utf-8?B?NVoyZkN1bGRPODcvT3pxdG01SnYvclZ6RXBYTG1kYzliTHBPZHZzNmxyMVpV?= =?utf-8?B?clJSUjVJSi9NUFNjZXVYbUJERDVsSmx5WnVtNHRweVpPRnBuZXM4VHRXUVNW?= =?utf-8?B?elFTZERzb1NMMzkzdUltMkRFU01zZnZKQnBTLzhkZ3ZlQ0U0UEpsMDNqMGRJ?= =?utf-8?B?UmNidEh6ZnJwKzNzM0FITTNTWlprR0RmRHh1TVdZTlJaeW1Rd2YzNEFtVUhp?= =?utf-8?B?eDZ5aUx0bXk2K2xlQTVFZ3UxeXphc2l1SVQwTlc2QmNNUEpHNDk3OVhNMWZK?= =?utf-8?B?ckJYY29sbW1tZnpIbHE1cjJnWkYvMS9ZRktWL3YxSHVGUUE5Z0ZvUlpVcUxF?= =?utf-8?B?RFZPU1ZwTldPVFg2aDVXR29DZHlBMFBCdm5XZHZLWWJTYjJUM1pXRjBiV0t6?= =?utf-8?B?OG1yZkRkZDhlVXZ2RWl6dTlTeWR3L2VoTExNd0s0YVc3QUhaenNCMlJlNE5u?= =?utf-8?B?dVZHN1FmeGpaVGJXU1EvQzNJMjY1cisreUtmWFYzRFJFWGxNeDRKbW1GWXRi?= =?utf-8?B?Y1Zaa2hvc0M4SEFVaFRWdk1wY05US0xyYXBUWEw5czllZGVMZThTZWpNS0tv?= =?utf-8?Q?zV+brcWQ85/p1RKzG2vtOY9lfzSW1Da9PVnnxRm?= X-OriginatorOrg: labn.net X-MS-Exchange-CrossTenant-Network-Message-Id: 4fb85dc0-d1ac-4384-3488-08d8e305697e X-MS-Exchange-CrossTenant-AuthSource: BL0PR14MB3779.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 14:12:45.8499 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: eb60ac54-2184-4344-9b60-40c8b2b72561 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Mi7Z3XM8WYc0MJEBpa1YJHv/iws+tOzqIMpnv5ajZ3aFGEgDq/rymueE4TIVOeYRsDMiaIQ2Ixkt4NVWFwooyA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR14MB3407 Archived-At: Subject: Re: [Detnet] session info X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 14:12:52 -0000 Hi, our second session start soon: NOTE: the meetecho URL for this meeting is: https://meetings.conf.meetecho.com/ietf110/?group=detnet&short=&item=2 the audio stream is at: http://mp3.conf.meetecho.com/ietf110/detnet/2.m3u note taking (towards middle): https://codimd.ietf.org/notes-ietf-110-detnet Lou On 3/8/2021 9:29 AM, Lou Berger wrote: > Reminder, we're about to get started for our first session... > > Materials: https://datatracker.ietf.org/meeting/110/session/detnet > Note taking:    https://codimd.ietf.org/notes-ietf-110-detnet > Jabber:    http://jabber.ietf.org/logs/detnet > WG ICS: https://datatracker.ietf.org/meeting/upcoming.ics?filters=detnet > > Session ICS: https://datatracker.ietf.org/meeting/110/session/28645.ics > Meetecho: > https://meetings.conf.meetecho.com/ietf110/?group=detnet&short=&item=1 > Audio stream: http://mp3.conf.meetecho.com/ietf110/detnet/1.m3u > > > > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://www.ietf.org/mailman/listinfo/detnet From nobody Tue Mar 9 09:05:22 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 182203A13ED; Tue, 9 Mar 2021 09:05:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.869 X-Spam-Level: X-Spam-Status: No, score=-0.869 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P-fvfGEhpVvK; Tue, 9 Mar 2021 09:05:13 -0800 (PST) Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:40]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9E74E3A13AA; Tue, 9 Mar 2021 09:05:13 -0800 (PST) Received: from faui48f.informatik.uni-erlangen.de (faui48f.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:52]) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTP id 8970654802F; Tue, 9 Mar 2021 18:05:05 +0100 (CET) Received: by faui48f.informatik.uni-erlangen.de (Postfix, from userid 10463) id 816CE440166; Tue, 9 Mar 2021 18:05:05 +0100 (CET) Date: Tue, 9 Mar 2021 18:05:05 +0100 From: Toerless Eckert To: Brian E Carpenter Cc: Tom Herbert , "Yangfan (IP Standard)" , DetNet WG , 6man WG , "draft-geng-6man-redundancy-protection-srh@ietf.org" Message-ID: <20210309170505.GA63862@faui48f.informatik.uni-erlangen.de> References: <3fdc1006788e47e59cfb8dcc03e9bce6@huawei.com> <9ff8cb12-d23d-74c2-fea7-900d1d4e2974@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9ff8cb12-d23d-74c2-fea7-900d1d4e2974@gmail.com> User-Agent: Mutt/1.10.1 (2018-07-13) Archived-At: Subject: Re: [Detnet] Flow Identification in IPv6 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 17:05:16 -0000 How about taking a bit from the "reserved" field in the draft to indicate if/when the IPv6 header flow label is being used as the flow identification field. That would make using the flow label field an option for the operator to decide. For example, i could see not wanting to use the flow label field if it's pushed down from encapulated IPv6 traffic and used already for some other purpose than PREOF. Cheers Toerless On Tue, Mar 09, 2021 at 05:03:22PM +1300, Brian E Carpenter wrote: > On 09-Mar-21 05:31, Tom Herbert wrote: > > On Mon, Mar 8, 2021 at 2:38 AM Yangfan (IP Standard) > > wrote: > >> > >> Hi Greg, > >> > >> Literally speaking, IPv6 Flow Label could be used to identify a specific flow needing redundancy protection in SRv6 data plane. But I may have concerns that flow label cannot be protected to be unmodified en route. A modified flow label can be a disaster for the traffics which are seeking for deterministic forwarding, not only this flow, also affecting other flows using redundancy protection. And with several security issues mentioned in RFC6437, I doubt if it is a good idea to user IPv6 Flow Label. > >> > >> Just my 2cents opinion, how do you and other people see it? > >> > > > > If this is to be used in a SRv6 domain which is itself a limited > > domain, then I think the problems you mention aren't as much of a > > concern since flow label would be used in a controlled environment. > > The upside of using flow label is that it's already in the IPv6 > > header, it can be consumed by non-SRv6 devices, and putting the same > > information in TLVs incurs the overhead and cost of processing TLVs in > > the critical datapath. > > The main downside is that it cannot convey any semantics and there is > a rule about how its value is created. It's no more at risk of being > modified than any other unauthenticated header field, so I agree that > within an SRV6 domain malicious modification doesn't seem like a > big risk. An attacker who could do that could do any kind of damage > they wanted. > > Regards > Brian > > > > > > Tom > > > >> > >> > >> Regards, > >> > >> Fan > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> ?????????: Greg Mirsky [mailto:gregimirsky@gmail.com] > >> ????????????: 2021???3???7??? 4:20 > >> ?????????: draft-geng-6man-redundancy-protection-srh@ietf.org > >> ??????: 6man WG ; DetNet WG ; Greg Mirsky > >> ??????: Flow Identification in IPv6 > >> > >> > >> > >> Dear Authors, > >> > >> thank you for bringing your proposal to the discussion. I agree with your view that the explicit routing enabled by SRv6 creates an environment where PREOF can be used. And, as we know, The PREOF may be used in a DetNet domain to lower packet loss ratio and provide bounded latency. > >> > >> After reading the draft, I've got a question for you. What do you see as the difference between the IPv6 Flow Label per RFC 6437 and the Flow Identification field in the TLV proposed in the draft? Could the IPv6 Flow Label be used to identify the flow for the PREOF? > >> > >> > >> > >> Regards, > >> > >> Greg > >> > >> -------------------------------------------------------------------- > >> IETF IPv6 working group mailing list > >> ipv6@ietf.org > >> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 > >> -------------------------------------------------------------------- > > > > -------------------------------------------------------------------- > > IETF IPv6 working group mailing list > > ipv6@ietf.org > > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 > > -------------------------------------------------------------------- > > > > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://www.ietf.org/mailman/listinfo/detnet -- --- tte@cs.fau.de From nobody Fri Mar 19 06:35:45 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B39C3A13DD for ; Fri, 19 Mar 2021 06:35:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.348 X-Spam-Level: X-Spam-Status: No, score=-2.348 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fh8LC5VCM15g for ; Fri, 19 Mar 2021 06:35:41 -0700 (PDT) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2057.outbound.protection.outlook.com [40.107.20.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5B1B63A13B6 for ; Fri, 19 Mar 2021 06:35:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H814UessUOoLRqL6Zoc2dUoAMfDO8Qo6t+BrTVny53e4qMdi0YnmNx/9LN72brIPKzbttYb16KH6O5p6ATpRpMkW2JHMpLHV721VDfnpWMQCrN8+egJbSEqjX9otBF8Rm1CmBXWYiQlKvEJ4EXPxh/4YehNQI//Zb0Vl2kGwV1/JhjZv51t0JDHth7nawQzVpOL4A36n7E4d33ugQU0dLulZFtFPtIcRei6mCvWJA3RiDIw3ntNWku2MzPK2w04BAHLBO0NjtNmQmyo2YP+j7dCVAg/Paki3nBFuu/Kq0OeyfXAAPcKbKy+PzIhJrenKVy6hu8K04lItq15Nhi4gIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F1QvMH6Z8r62vdqxCG0fYBhvB8Fl5At4e9aJG/NmKp8=; b=Uf75kado/aFOejdDZlfc5vns/ITo4ttMwiM+Hl/ENKtwatcQ5oHjMVF4Jt/bdAj4cJMOA7xvzkJvYi6uPt3FhpxbjnAHPMbw61iGnPSpi3YrnMnKKgywWtOeyHD5p8CkAPXsbQYGbaTGGSAlgjFRZqj/3/zO5ZcMOcJ7v34E5egJkgDV80f2qEvZ6bOVtWApttWrMNOAPOO+dxbSe8NdLhSTqfCRCXEGoUgRW/LAbxfJzvhBhOCYA3b5/nYK8tam3Pw2juHjqcR/922sHUo7rkcgk2CFqgEB8Y9VeoOElFypWlx/S8L3+hQWgpeZq+r1HLoAb2CFrMgXMpLeuFikSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F1QvMH6Z8r62vdqxCG0fYBhvB8Fl5At4e9aJG/NmKp8=; b=NRG/rkPoPTVuqbNb29/OoBtPVio54BbRjZ12xYrt+8CU3zTrW+G9pf5vZNNhE2DrQ//PUzGbMVVQ/dAHk0ZKF+CSitJRWhqeyVVMM1S1OxmIxNQDaV8b8q4fxCcpm5Ncwb12mAMNl+BghsFFbBZ82VQ4hQRTxYCetwcWWm6CNfU= Received: from AM9PR07MB7204.eurprd07.prod.outlook.com (2603:10a6:20b:2cb::15) by AM0PR07MB5969.eurprd07.prod.outlook.com (2603:10a6:208:10f::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10; Fri, 19 Mar 2021 13:35:37 +0000 Received: from AM9PR07MB7204.eurprd07.prod.outlook.com ([fe80::60f8:2297:7367:cf31]) by AM9PR07MB7204.eurprd07.prod.outlook.com ([fe80::60f8:2297:7367:cf31%6]) with mapi id 15.20.3977.016; Fri, 19 Mar 2021 13:35:37 +0000 From: Janos Farkas To: DetNet WG Thread-Topic: IETF 110 draft minutes uploaded Thread-Index: AdccxExK+4v7nS10THqHoYz+EWJ/yA== Date: Fri, 19 Mar 2021 13:35:36 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [2a02:ab88:36c1:cb00:4c8b:31bb:8892:bb9e] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b235ddbf-f9e7-4952-d1fc-08d8eadbe140 x-ms-traffictypediagnostic: AM0PR07MB5969: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4U4CNm9RQ7GpbI5HisPTY2rgCcqMZPlmEZyZYIoebpT49UAXzmBVahKOeYDB/kAZclWp5pZeOW0oKIf3H5M/umC+yEXw9pWjgoYrAHqwT8XGmL752r/QwgsVwi1eRqlF7RLnJvK5XYv+VOk0D18A6OQo1Fr/+WA1Uyu3Ig0lPv3VQ/4+QfoTdHY5W9k+6eK4m8LDKXXINWv8UL7+1f9gvxXNJgrFzCaCaSIuJxxavW0yHAddqISIkBdCzxLcvNQzWpDgqm+2dljDqRFU9F0Jj7/Xr7Yc/wASMbDHiSAz4jmOpKr2ek1fRAMwSHI0TFNCJ95c0YYDljwYokHsTVEqCG2cWOIQTa14S5mG2No2LSd7vO0D24VDANBC63Ym+mcsqGu/cuiAniER57cYc8SGCqnPlqa3/j/JRZ6Ou7aglPJ3EPl/vNVsIBDKFEmGagabFwsJEcy2JAenzhgY+aHdanQcBZ2SqY3LYxnT/PmAT1qfXlc6PJVPmjFLuptmQ47E5c/eDoRcsXOo1QjQU6xvXZ2/lmBAraeDUth4/FRUdvsCGqIAUzzBmUsArc+ref4ADsb8ojZldYOeWlPxYHRjIBw9peiUiqJEiw3iFh8DMlbhVDV2kgcTPNCf3LCmlEuAp/FYb1kl+1HQNwgVxnpK5pjYM29tCrV/4yWvP4RBZbBPi0noA97ryOnSGkgtMX+rnviA6pAc+TvBjZZe+V7tvuVhuso2aQAFJj8aXePDkEo= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM9PR07MB7204.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(366004)(136003)(396003)(376002)(346002)(9686003)(55016002)(478600001)(966005)(5660300002)(64756008)(166002)(52536014)(7696005)(6916009)(66556008)(38100700001)(66946007)(186003)(76116006)(66476007)(66446008)(8936002)(2906002)(8676002)(558084003)(6506007)(71200400001)(86362001)(33656002)(9326002)(316002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?OaxFqKyi1I6TtTJ+4gvJvO/JNqnNlSxj1wjSYTCPrPT0wNPZTlCdFf10Lm21?= =?us-ascii?Q?HzH1VdtJfwoQ4kzbTpBM13qa01ShTt02M9bd5vHs+2L1zUN3Trj7Q15Ivk2s?= =?us-ascii?Q?QUsF2bR8MrartxS4AlfN6VMIQOYhUS++k1GFwFh9IpSiiLaNPPD5AY3JvQ3i?= =?us-ascii?Q?mDAef/2jRb0oIyj2+3jyHf303xLeLTb3rt/jfT8PrY/1gvhtaXt0FBlzBnGV?= =?us-ascii?Q?5aDsq7c9fEmV76oy9XKZudH2mlJLslX4+U4ZwT14R15Ayaq9/1G8LBqafZdg?= =?us-ascii?Q?zOL+OZJaaQ/vNGLSqjMpUc/BbkOSV698tvhz2G9jAmuS6I0QDUXrwJscw0Vn?= =?us-ascii?Q?t48LShE7LMvBn9kBcwv+R9zsZEUtPuP5rY72n9c6aDc9OnUEv0fIDt47lRax?= =?us-ascii?Q?Bje3b4+3PcyrOZ8sF/p4plO5ReypE/3U9uA+qG7UOkWyEM+qapar7Ce6UdAC?= =?us-ascii?Q?veYOK2b1eer4TdcHjZlQoOpVmm4asfd2/xxCtmVKBk33eRCUfjWpFhAqEXON?= =?us-ascii?Q?YagdO0aMbjukaUlQ/fAJO6zIi6XlIimGsaWvpeVl8NyHJmtsQWpMNVJTRg8f?= =?us-ascii?Q?XkoXA37XQMhOV7FVBJHhSej02sIT444ZXq5RUdSE8evsT7ryWHkMAVXHZdMp?= =?us-ascii?Q?Z3BrivyB82PdvdZIUdr8giclOO71s6m2gSmlcpDChPWjHooTophODtG+LpKj?= =?us-ascii?Q?/DzRYDO0Mg2cJ/zRE+EZZ/IWVcb0MEEk+FdjjL7Y/nQhCkUuX89Dn2XQEGiN?= =?us-ascii?Q?XD+p/hAwxPt0lPmWZU/brkblY6RXzwF3tIUGkPhyZ/spVBA2k8jDfN7MIODI?= =?us-ascii?Q?o6Pp93Z2iqDqXnhMK2aZwUj1vFZW7PIyhctg/B6M+0Ym2b7GhxXk6uSQMebf?= =?us-ascii?Q?0T69OuNAoBp25zs6thAmfTuJX+TXMbUBNy0dFwEretyvjjD96ke42orPzOz6?= =?us-ascii?Q?EDoHdhQeZCHz9fsMicxAIfx6YFHARWZmzOblYSjuAIFqiagfj/xHMFPwx1zL?= =?us-ascii?Q?q5POObSrK2yynyHppamSy89wrCEmJePxfbAqlBiYZJr/Gw2MIaIwJGhldm6K?= =?us-ascii?Q?OImSlymQi0ALojYxGNQw140aBDAZDfKuIzah3bTgqa9IYNYTSVs4kQfOD0dW?= =?us-ascii?Q?pQeFt0WJ/jB0J4/HQCLHjqV/QDA+/WEvHHSNq+SusmyIaFwTAyZaxkyjQ51O?= =?us-ascii?Q?fmWiXSbTotJ59VcMEeAfWCHs8IPHNdkjWtZrK6NE2S6U5YFjJ5OreBFHRrZ1?= =?us-ascii?Q?AK/xiEt3/voB0kHYGIJOTrb2MmgmG4bZyrQNYj7zROu7ro6HEvZFD+ALotQP?= =?us-ascii?Q?/uoFx1OY85zkod82HQdNMjRMS+xwOdZCVNQKN6ic8vA+SjEdbzM6A44e2Fcm?= =?us-ascii?Q?r4ThaVRFog5wv7rvI44eI/G3BcYM?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM9PR07MB72042FCD9D08B45C048C7924F2689AM9PR07MB7204eurp_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM9PR07MB7204.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b235ddbf-f9e7-4952-d1fc-08d8eadbe140 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2021 13:35:36.9440 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: d6cepVgzVBbJV4eWT+9rWqTlJTy58pHo5/qur/imXkYaHESxjTxk98izpVUfP2j0WTf1fbBwKAuZ9Zmt7h+P0mKCCEgnSgbKCr++pwDM/eI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB5969 Archived-At: Subject: [Detnet] IETF 110 draft minutes uploaded X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2021 13:35:43 -0000 --_000_AM9PR07MB72042FCD9D08B45C048C7924F2689AM9PR07MB7204eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Many thanks to Ethan Grossman (WG Secretary) for writing the minutes! Draft minutes have been posted: https://datatracker.ietf.org/meeting/110/materials/minutes-110-detnet Please send comments/corrections to the list. Thank you, Janos (Lou and Ethan) --_000_AM9PR07MB72042FCD9D08B45C048C7924F2689AM9PR07MB7204eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

Many thanks to Ethan Grossman (WG Secretary) for wri= ting the minutes!

Draft minutes have been posted:

https://datatracker.ietf.org/meeting/110/mate= rials/minutes-110-detnet

Please send comments/corrections to the list.

Thank you,

Janos (Lou and Ethan)

--_000_AM9PR07MB72042FCD9D08B45C048C7924F2689AM9PR07MB7204eurp_-- From nobody Fri Mar 19 07:29:55 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 052AF3A1698; Fri, 19 Mar 2021 07:29:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.097 X-Spam-Level: X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PqnFRaj-WN-R; Fri, 19 Mar 2021 07:29:53 -0700 (PDT) Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8A44A3A1695; Fri, 19 Mar 2021 07:29:53 -0700 (PDT) Received: by mail-qk1-x72d.google.com with SMTP id q3so3013674qkq.12; Fri, 19 Mar 2021 07:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=gGX3iGOJGZHJDHsNfy3cXUjXHUf1ie/xhxEkJI6haqE=; b=YTdv45VVKo/Uk/6Ne2Z/yX4Wuyw1DmvGZunVpSiBf6VYBkEz/S2M1K+EdmqzvPi/kZ hz62gH+Khq3dDBWaLgrTgJP4wY/bsc1AsjvMJ/JLYA+1SK49tJ8bE+FjlWlQZYlGflZX dc54pMing3ZaZrKLNWTS5HyepZyZa/kSup0RacUp/DxvqjOo5PrWovBn4cpswlHeO3sn UWfok+itNJe32hg1u9PWu9mOVQkN9nCxh7AQIGJv3VfrZ9VynufuTiwLVlNz8lPAYUCV McF4LMqavPma6Q0uyZS8PORQdJGlNZF44sGbU208BT1cpvuXwNl6t3mPYnI1SzTGgyqb eQBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=gGX3iGOJGZHJDHsNfy3cXUjXHUf1ie/xhxEkJI6haqE=; b=RuvfFo2S2RXYH10kgUmG36sOxsOC7Yj/4w1J0CqERp5qY+5tFoHqtmwcGafPfig2sX i4MWxjsAZIb5B/T2bz8zluklnbmWlYc3ro9p52L/j26pWA2zxzFrpEtwgNHFkVD4Ycjj 6Z1Gme1ppH/xdaluMRvQ/B8hROJq0ma2ciw9B8F0/R6LK3dGv4/SXaarLHGW99n4X6JH BjmEV/gkYHoCwBtc2WUdZtSj3fJZQrzzmsT5kF27bGRmenfdru8ETAWuOtYTN0e/AuQG pBP/l5azqr6hpAQFzKBED6UBNRBj0W6otFj6dEl01BLvadM0MgxpC6x9facAVcol30Xl 5OSQ== X-Gm-Message-State: AOAM533NK+xe60yJmTKp4Hlb0nlsD7SsaOMm3WtPXHTCGc++15t1X1vQ QhJvF+ZWFRXRbxtM9wgIWL/Xag588Oa67WEk374VxFD6g2U= X-Google-Smtp-Source: ABdhPJzRA9rUt0WGFW4uGM0J985moMbSZ95t4cX0RWMm+lOTHRhke2iCF0LKE5vG5zxsc8LqmKEIF7tYAYUt7242c48= X-Received: by 2002:a37:6491:: with SMTP id y139mr9695028qkb.483.1616164189929; Fri, 19 Mar 2021 07:29:49 -0700 (PDT) MIME-Version: 1.0 From: "Andrew G. Malis" Date: Fri, 19 Mar 2021 10:29:34 -0400 Message-ID: To: pals@ietf.org Cc: detnet WG , SPRING WG List , mpls Content-Type: multipart/alternative; boundary="00000000000070831e05bde48c6f" Archived-At: Subject: [Detnet] Joint PALS/MPLS/DetNet/SPRING minutes uploaded X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2021 14:29:55 -0000 --00000000000070831e05bde48c6f Content-Type: text/plain; charset="UTF-8" Many thanks to Dave Sinicrope (WG Secretary) for taking the minutes! Draft minutes have been posted: https://datatracker.ietf.org/meeting/110/materials/minutes-110-pals Please send comments and corrections to the PALS list only. Cheers, Andy --00000000000070831e05bde48c6f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Many thanks to Dave Sinicrope (WG Secretary) for taking th= e minutes!

Draft minutes have been posted:
https://datatrack= er.ietf.org/meeting/110/materials/minutes-110-pals

Please send c= omments and corrections to the PALS list only.

Cheers,
Andy
<= br>
--00000000000070831e05bde48c6f-- From nobody Mon Mar 22 00:55:33 2021 Return-Path: X-Original-To: detnet@ietf.org Delivered-To: detnet@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E0AA3A16F0; Mon, 22 Mar 2021 00:55:32 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit From: internet-drafts@ietf.org To: Cc: detnet@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 7.27.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: detnet@ietf.org Message-ID: <161639973214.8758.5107802709738409866@ietfa.amsl.com> Date: Mon, 22 Mar 2021 00:55:32 -0700 Archived-At: Subject: [Detnet] I-D Action: draft-ietf-detnet-bounded-latency-03.txt X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 07:55:32 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Deterministic Networking WG of the IETF. Title : DetNet Bounded Latency Authors : Norman Finn Jean-Yves Le Boudec Ehsan Mohammadpour Jiayi Zhang Balázs Varga János Farkas Filename : draft-ietf-detnet-bounded-latency-03.txt Pages : 28 Date : 2021-03-22 Abstract: This document references specific queuing mechanisms, defined in other documents, that can be used to control packet transmission at each output port and achieve the DetNet qualities of service. This document presents a timing model for sources, destinations, and the DetNet transit nodes that relay packets that is applicable to all of those referenced queuing mechanisms. Using the model presented in this document, it should be possible for an implementor, user, or standards development organization to select a particular set of queuing mechanisms for each device in a DetNet network, and to select a resource reservation algorithm for that network, so that those elements can work together to provide the DetNet service. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-detnet-bounded-latency/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-detnet-bounded-latency-03 https://datatracker.ietf.org/doc/html/draft-ietf-detnet-bounded-latency-03 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-detnet-bounded-latency-03 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Mon Mar 22 08:31:28 2021 Return-Path: X-Original-To: detnet@ietf.org Delivered-To: detnet@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 56BC63A089C; Mon, 22 Mar 2021 08:31:26 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit From: internet-drafts@ietf.org To: Cc: detnet@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 7.27.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: detnet@ietf.org Message-ID: <161642708629.11371.9258735155273440053@ietfa.amsl.com> Date: Mon, 22 Mar 2021 08:31:26 -0700 Archived-At: Subject: [Detnet] I-D Action: draft-ietf-detnet-bounded-latency-04.txt X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 15:31:26 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Deterministic Networking WG of the IETF. Title : DetNet Bounded Latency Authors : Norman Finn Jean-Yves Le Boudec Ehsan Mohammadpour Jiayi Zhang Balázs Varga János Farkas Filename : draft-ietf-detnet-bounded-latency-04.txt Pages : 28 Date : 2021-03-22 Abstract: This document references specific queuing mechanisms, defined in other documents, that can be used to control packet transmission at each output port and achieve the DetNet qualities of service. This document presents a timing model for sources, destinations, and the DetNet transit nodes that relay packets that is applicable to all of those referenced queuing mechanisms. Using the model presented in this document, it should be possible for an implementor, user, or standards development organization to select a particular set of queuing mechanisms for each device in a DetNet network, and to select a resource reservation algorithm for that network, so that those elements can work together to provide the DetNet service. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-detnet-bounded-latency/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-detnet-bounded-latency-04 https://datatracker.ietf.org/doc/html/draft-ietf-detnet-bounded-latency-04 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-detnet-bounded-latency-04 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Mon Mar 22 08:33:56 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 639C53A08BE for ; Mon, 22 Mar 2021 08:33:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.199 X-Spam-Level: X-Spam-Status: No, score=-0.199 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=epfl.ch Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g_62M35cccBz for ; Mon, 22 Mar 2021 08:33:48 -0700 (PDT) Received: from smtp0.epfl.ch (smtp0.epfl.ch [IPv6:2001:620:618:1e0:1:80b2:e058:1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 082083A08C5 for ; Mon, 22 Mar 2021 08:33:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=epfl.ch; s=epfl; t=1616427222; h=From:To:CC:Date:Message-ID:Content-Type:MIME-Version; bh=3jbDC4wpPI1Qp3iaC+6IA4flyiUCuLgf5PUraIX65TI=; l=22932; b=Jxc4UYPvy1YJ2raAhx8ZhU+3/NbYE9lDsdlp9aOWeEmCzwa7N2ZMIrSm/fUfsWpmU CpzHzHZipVZBZ6z78QLPSXT90FWKl2l9rigdQaBFQYfXVTp392RJV93nxOz4Jh775 nI3vYp/Yd1zS0Owk0VOvk/lC2gISlqCP7B1F2QXBc= Received: (qmail 23448 invoked by uid 107); 22 Mar 2021 15:33:42 -0000 Received: from ax-snat-224-174.epfl.ch (HELO ewa05.intranet.epfl.ch) (192.168.224.174) (TLS, AES256-GCM-SHA384 cipher) by mail.epfl.ch (AngelmatoPhylax SMTP proxy) with ESMTPS; Mon, 22 Mar 2021 16:33:42 +0100 X-EPFL-Auth: Meo08Wsu7Q1EAoj5BEpnCErAqt3LtafvkFC/dD8V0R64R0ygZDc= Received: from ewa02.intranet.epfl.ch (128.178.224.159) by ewa05.intranet.epfl.ch (128.178.224.174) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Mon, 22 Mar 2021 16:33:42 +0100 Received: from ewa02.intranet.epfl.ch ([fe80::8939:e5e2:d561:d768]) by ewa02.intranet.epfl.ch ([fe80::8939:e5e2:d561:d768%4]) with mapi id 15.01.2106.013; Mon, 22 Mar 2021 16:33:42 +0100 From: Mohammadpour Ehsan To: Lou Berger CC: DetNet WG , "draft-ietf-detnet-bounded-latency@ietf.org" Thread-Topic: [Detnet] WG Last Call: draft-ietf-detnet-bounded-latency-02 Thread-Index: AQHW+8MLF9W102eQAk2Yhw/0iYu+aKpjNeAAgC0jTQA= Date: Mon, 22 Mar 2021 15:33:42 +0000 Message-ID: <8608941F-AD44-4098-83DF-416FBD2DFEA3@epfl.ch> References: <371c2363-4f15-b32f-4c96-483a31ae1c77@labn.net> In-Reply-To: <371c2363-4f15-b32f-4c96-483a31ae1c77@labn.net> Accept-Language: en-US, fr-CH Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.179.253.144] Content-Type: multipart/alternative; boundary="_000_8608941FAD44409883DF416FBD2DFEA3epflch_" MIME-Version: 1.0 Archived-At: Subject: Re: [Detnet] WG Last Call: draft-ietf-detnet-bounded-latency-02 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 15:33:54 -0000 --_000_8608941FAD44409883DF416FBD2DFEA3epflch_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 RGVhciBXRywNCg0KV2Ugc3VibWl0dGVkIGEgbmV3IHZlcnNpb24gb2YgdGhlIEJvdW5kZWQgTGF0 ZW5jeSBkcmFmdCB0aGF0IGFkZHJlc3NlcyB0aGUgY29tbWVudHMgcmVjZWl2ZWQgaW4gdGhlIGxp c3QgKGh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvZG9jL2h0bWwvZHJhZnQtaWV0Zi1kZXRu ZXQtYm91bmRlZC1sYXRlbmN5LTA0PGh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1p ZXRmLWRldG5ldC1ib3VuZGVkLWxhdGVuY3ktMDM+KTsgaXQgaXMgbm93IHJlYWR5IGZvciBzdWJt aXNzaW9uIHRvIHRoZSBJRVNHIGZvciBwdWJsaWNhdGlvbi4NCg0KVGhhbmtzIExvdSBmb3IgaGlz IGNvbW1lbnRzLiBJbiB0aGUgbmV3IHZlcnNpb24sIHdlIGFkZHJlc3MgdGhlIGNvbW1lbnRzIGFz IGZvbGxvd3M6DQoNCi0gQWJzdHJhY3QNCkkgdGhpbmsgdGhlIGFic3RyYWN0IHNob3VsZCB0byBi ZSB1cGRhdGVkIHRvIG1hdGNoIHRoZSBjdXJyZW50IGNvbnRlbnQgb2YgdGhlIGRvY3VtZW50LiBJ IHN1Z2dlc3QganVzdCBjb3B5aW5nIHRoZSA0dGggYW5kIDV0aCBwYXJhZ3JhcGhzIGZyb20gdGhl IEludHJvZHVjdGlvbiBzZWN0aW9uLg0KDQoNClRoYW5rcyBmb3IgdGhlIHN1Z2dlc3Rpb24uIFRo ZSBuZXcgdmVyc2lvbiBpcyB1cGRhdGVkIGFjY29yZGluZ2x5Lg0KDQotIGdlbmVyYWwgY29tbWVu dDogY2xhc3NlcyBvZiAoRGV0TmV0KSBTZXJ2aWNlDQoNClRoZSBkb2N1bWVudCBzdGF0ZXMgdGhh dCBpdCB1c2VzIHRlcm1pbm9sb2d5IGZyb20gUkZDODY1NSwgaXQgYWxzbyB1c2VzIHRoZSB0ZXJt cyAiQ2xhc3NlcyBvZiBEZXROZXQgU2VydmljZSIsICIgRGV0TmV0IENsYXNzZXMgb2YgU2Vydmlj ZSIgIGFuZCBvdGhlciBzaW1pbGFyIGZvcm1zIG9mIHRoZXNlIHRlcm1zLiAgUkZDODY1NSB1c2Vz IHRoaXMgdGVybSBpbiBvbmUgcGxhY2UgIiBDbGFzcy1vZi1TZXJ2aWNlIHNjaGVtZXMgKGUuZy4s IERpZmZTZXJ2KSIgYW5kIGFsc28gbWVudGlvbnMgImNsYXNzZXMgb2YgRGV0TmV0IGZsb3dzIiAg aW4gdHdvIHBsYWNlcy4gIFRoZSBkYXRhIHBsYW5lIGRvY3VtZW50cyBbUkZDODk2NF0gYW5kIFtS RkM4OTM0XSBhbHNvIGRlc2NyaWJlIENsYXNzIG9mIFNlcnZpY2UgaW4gdGhlIGNvbnRleHQgb2Yg RGlmZlNlcnYgKGFuZCBub3QgRGV0TmV0KS4NCg0KSW4gcmVhZGluZyB0aGUgZG9jdW1lbnQgaXQg c2VlbXMgdGhhdCBpdCBpcyBub3QgdXNpbmcgQ29TIGluIHRoZSB0eXBpY2FsIHdheSB1c2VkIGlu IHRoZSBJRVRGICBvciB0aGUgb3RoZXIgRGV0TmV0IFJGQ3MsIGFuZCBpcyBpbnRyb2R1Y2luZyBh IG5ldyBkZWZpbml0aW9uIGZvciBDb1MuICAgSUkgdGhpbmsgaGF2aW5nIGEgZG9jdW1lbnQgc3Bl Y2lmaWMgZGVmaW5pdGlvbiBvZiBDb1MgcHJvYmFibHkgaXNuJ3QgaGVscGZ1bC4gSXQgd291bGQg YmUgYmVzdCB0byB1c2UgZXhpc3RpbmcgRGV0TmV0IHRlcm1pbm9sb2d5LiAgSSBzdXNwZWN0ICAi ZmxvdyBhZ2dyZWdhdGUiIGlzIHRoZSBjbG9zZXN0LCBidXQgcGVyaGFwcyBJJ20gbWlzc2luZyBz b21ldGhpbmcuDQoNCg0KVGhhbmtzIGZvciB5b3VyIGNvbW1lbnQuIEluIHRoZSBuZXcgdmVyc2lv biwgd2UgdXNlZCB0aGUgdGVybSDigJxhZ2dyZWdhdGXigJ0gaW5zdGVhZCBvZiBjbGFzcyBpbiBz ZWN0aW9uIDMuMSBhbmQgY2xhc3NlcyBvZiBEZXROZXQgZmxvd3MuIFNlY3Rpb24gNi40IHVzZXMg dGhlIHRlcm0g4oCcdHJhZmZpYyBjbGFzc+KAnSBhcyBpbiBJRUVFODAyLjFRLg0KDQotIGFsaWdu bWVudCB3aXRoIHRoZSBEYXRhIFBsYW5lIFJGQ3MNCg0KVGhlIGxpc3RzIGluIHNlY3Rpb24gMy4x IGFuZCBpbiBzZWN0aW9uIDYuNCBhcmUgbm90IHdlbGwgYWxpZ25lZCB3aXRoIHRoZSBEYXRhIHBs YW5lIGRvY3VtZW50cy4gIFRoZXNlIHNlY3Rpb25zIHNob3VsZCBiZSBhbGlnbmVkIHdpdGggdGhl IHByb3Zpc2lvbmluZyBvZiBJUCBmbG93cywgYXMgc3VtbWFyaXplZCBpbiBTZWN0aW9uICA2IG9m IFJGQzg5MzksIGFuZCBNUExTIGFzIHN1bW1hcml6ZWQgaW4gU2VjdGlvbiA1IG9mIFJGQzg5NjQu DQoNClRoYW5rcyBmb3IgeW91ciBjb21tZW50LiBTZWN0aW9uIDMuMSBpcyBhYm91dCBmbG93IGFk bWlzc2lvbiBwYXJhZGlnbSBhbmQgbm90ICJmbG93IGNyZWF0aW9u4oCdLiBUaGUgdGl0bGUgaXMg Y2hhbmdlZCB0byDigJxmbG93IGFkbWlzc2lvbuKAnSB0byBhdm9pZCBwb3NzaWJsZSBjb25mdXNp b24uIEluIGZhY3QsIHRoaXMgc2VjdGlvbiBpcyBhbGlnbmVkIHdpdGggdGhlIERhdGFQbGFuZSBS RkNzIGFuZCBpbiBwYXJhbGxlbCB3aXRoIHByb3Zpc2lvbmluZyBvZiBJUCBmbG93cy4gV2UgbW9k aWZpZWQgc2VjdGlvbiA2LjQgdG8gZm9sbG93IHRoZSBEYXRhIHBsYW5lIGRvY3VtZW50cy4NCg0K DQotIG1pbm9yIGNvbW1lbnQgc2VydmljZSB2cyBmcmFtZSBwcmVlbXB0aW9uDQoNClNlY3Rpb24g My4xIHVzZXMgdGhlIHRlcm0gInVuLXByb3Zpc2lvbmluZyIsIEkgcmVhZCB0aGUgdGV4dCBhcyBt ZWFuaW5nICJzZXJ2aWNlIHByZWVtcHRpb24iLg0KDQpJbiB0aGUgcmVzdCBvZiB0aGUgZG9jdW1l bnQgcHJlZW1wdGlvbiByZWZlcnMgdG8gImZyYW1lIHByZWVtcHRpb24iIGFuZCB0aGlzIHNob3Vs ZCBiZSBjbGFyaWZpZWQuDQoNClRoYW5rcyBmb3IgdGhlIHN1Z2dlc3Rpb24uIEluIHRoZSBuZXcg dmVyc2lvbiwgd2UgdXNlIHRoZSB0ZXJtIOKAnHNlcnZpY2UgcHJlZW1wdGlvbuKAnSBhbmQgdXBk YXRlIHRoZSB0ZXJtIOKAnHByZWVtcHRpb27igJ0gdG8g4oCcZnJhbWUgcHJlZW1wdGlvbuKAnS4N Cg0KDQotIGlkIG5pdHMsIHBsZWFzZSBlbnN1cmUgdGhlIGRvY3VtZW50IHBhc3NlcyBpZG5pdHMg d2l0aG91dCBlcnJvcnMsIHNlZQ0KDQpodHRwczovL3d3dzYuaWV0Zi5vcmcvdG9vbHMvaWRuaXRz P3VybD1odHRwczovL3d3dy5pZXRmLm9yZy9hcmNoaXZlL2lkL2RyYWZ0LWlldGYtZGV0bmV0LWJv dW5kZWQtbGF0ZW5jeS0wMi50eHQNCg0KVGhhbmtzIGZvciB5b3VyIGNvbW1lbnQgYWJvdXQgaWRu aXRzLiBXZSBhZGRlZCB0aGUgbWlzc2luZyBzZWN0aW9ucywgaS5lLiwg4oCcc2VjdXJpdHkgY29u c2lkZXJhdGlvbnPigJ0gYW5kIOKAnElBTkEgY29uc2lkZXJhdGlvbnPigJ0gdG8gdGhlIGRyYWZ0 LiBBbHNvIHJlc29sdmVkIG90aGVyIGNvbW1lbnRzIGJ5IGlkbml0cy4NCg0KQmVzdCwNCkVoc2Fu DQoNCg0KDQoNCi0tDQpFaHNhbiBNb2hhbW1hZHBvdXINClBoRCBDYW5kaWRhdGUgYXQgU3dpc3Mg RmVkZXJhbCBJbnN0aXR1dGUgb2YgVGVjaG5vbG9neSAoRVBGTCkNCkVQRkwgRURJQyBQaEQgc3R1 ZGVudCByZXByZXNlbnRhdGl2ZQ0KSUMgSUlORkNPTSwgTENBMiwgSU5GIDAxMSwgU3RhdGlvbiAx NCwgMTAxNSBMYXVzYW5uZSwgU3dpdHplcmxhbmQNCmh0dHBzOi8vcGVvcGxlLmVwZmwuY2gvZWhz YW4ubW9oYW1tYWRwb3VyPGh0dHBzOi8vcGVvcGxlLmVwZmwuY2gvZWhzYW4ubW9oYW1tYWRwb3Vy P2xhbmc9ZW4+DQoNCg0KT24gMjEgRmViIDIwMjEsIGF0IDIzOjE1LCBMb3UgQmVyZ2VyIDxsYmVy Z2VyQGxhYm4ubmV0PG1haWx0bzpsYmVyZ2VyQGxhYm4ubmV0Pj4gd3JvdGU6DQoNCkFsbCwNCg0K VGhlIFdHIGxhc3QgY2FsbCBpcyBjb21wbGV0ZS4NCg0KQXV0aG9ycywNCg0KUGxlYXNlIGJyaW5n IGFueSByZXNvbHV0aW9ucyB0byBjb21tZW50cyByZWNlaXZlZCB0byB0aGUgbGlzdC4gT25jZSBh bGwgY29tbWVudHMgYXJlIGFkZHJlc3NlZCBwbGVhc2UgdXBkYXRlIHRoZSBkcmFmdCBhbmQgbGV0 IHRoZSBXRyBrbm93IHRoYXQgaXQgaXMgcmVhZHkgZm9yIHN1Ym1pc3Npb24gdG8gdGhlIElFU0cg Zm9yIHB1YmxpY2F0aW9ucy4NCg0KVGhhbmsgeW91IQ0KDQpMb3UNCg0KT24gMi81LzIwMjEgODoz MCBBTSwgTG91IEJlcmdlciB3cm90ZToNCkFsbCwNCg0KVGhpcyBzdGFydHMgd29ya2luZyBncm91 cCBsYXN0IGNhbGwgb24gZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTAyDQpodHRw czovL2RhdGF0cmFja2VyLmlldGYub3JnL2RvYy9kcmFmdC1pZXRmLWRldG5ldC1ib3VuZGVkLWxh dGVuY3kvDQoNClRoZSB3b3JraW5nIGdyb3VwIGxhc3QgY2FsbCBlbmRzIG9uIEZlYnJ1YXJ5IDE5 dGguDQpQbGVhc2Ugc2VuZCB5b3VyIGNvbW1lbnRzIHRvIHRoZSB3b3JraW5nIGdyb3VwIG1haWxp bmcgbGlzdC4NCg0KUGxlYXNlIG5vdGUsIHRoZXJlIHdhcyBhbiBJUFIgZGlzY2xvc3VyZSBhZ2Fp bnN0IHRoaXMgZG9jdW1lbnQNCnN1Ym1pdHRlZCBvbiBKYW51YXJ5IDE0LCAyMDIxLCBzZWUgaHR0 cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9pcHIvNDU4MS8NCldoaWxlIHRoaXMgZGlzY2xvc3Vy ZSBjYW1lIHZlcnkgbGF0ZSBpbiB0aGUgZG9jdW1lbnQgbGlmZSBjeWNsZSwNCml0IGFwcGVhcnMg dGhlIGZpbGluZyB3YXMgYWxzbyByZWxhdGl2ZWx5IHJlY2VudCAoMjAxOS0xMC0xNikgYW5kDQph ZnRlciB0aGUgcHJlIGFkb3B0aW9uIElQIGNhbGw6DQpodHRwczovL21haWxhcmNoaXZlLmlldGYu b3JnL2FyY2gvbXNnL2RldG5ldC8xbXdZZGFkRHpUTHVkY3VINVQ1TzdSX0tsRHMNCg0KUG9zaXRp dmUgY29tbWVudHMsIGUuZy4sICJJJ3ZlIHJldmlld2VkIHRoaXMgZG9jdW1lbnQNCmFuZCBiZWxp ZXZlIGl0IGlzIHJlYWR5IGZvciBwdWJsaWNhdGlvbiIsIGFyZSB3ZWxjb21lIQ0KVGhpcyBpcyB1 c2VmdWwgYW5kIGltcG9ydGFudCwgZXZlbiBmcm9tIGF1dGhvcnMuDQoNClRoYW5rIHlvdSwNCkxv dSAoRGV0TmV0IENvLUNoYWlyICYgZG9jIFNoZXBoZXJkKQ0KDQoNCg0KX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCmRldG5ldCBtYWlsaW5nIGxpc3QNCmRl dG5ldEBpZXRmLm9yZw0KaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9kZXRu ZXQNCg0K --_000_8608941FAD44409883DF416FBD2DFEA3epflch_ Content-Type: text/html; charset="utf-8" Content-ID: <78C166B037161B48AFEC269D3CDC891C@intranet.epfl.ch> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0 ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHlsZT0id29yZC13 cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFm dGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9IndvcmQt d3JhcDogYnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBh ZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBjbGFzcz0iIj5EZWFyIFdHLDwvZGl2 Pg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+V2Ug c3VibWl0dGVkIGEgbmV3IHZlcnNpb24gb2YgdGhlIEJvdW5kZWQgTGF0ZW5jeSBkcmFmdCB0aGF0 IGFkZHJlc3NlcyB0aGUgY29tbWVudHMgcmVjZWl2ZWQgaW4gdGhlIGxpc3QgKDxhIGhyZWY9Imh0 dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1pZXRmLWRldG5ldC1ib3VuZGVkLWxhdGVu Y3ktMDMiIGNsYXNzPSIiPmh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvZG9jL2h0bWwvZHJh ZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTA0PC9hPik7DQogaXQgaXMgbm93IHJlYWR5 IGZvciBzdWJtaXNzaW9uIHRvIHRoZSBJRVNHIGZvciBwdWJsaWNhdGlvbi48L2Rpdj4NCjxkaXYg Y2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPg0KPGRpdiBjbGFz cz0iIj5UaGFua3MgTG91IGZvciBoaXMgY29tbWVudHMuIEluIHRoZSBuZXcgdmVyc2lvbiwgd2Ug YWRkcmVzcyB0aGUgY29tbWVudHMgYXMgZm9sbG93czo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+PGJy IGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPg0KPGJsb2NrcXVvdGUgdHlwZT0iY2l0 ZSIgY2xhc3M9IiI+LSBBYnN0cmFjdDxiciBjbGFzcz0iIj4NCkkgdGhpbmsgdGhlIGFic3RyYWN0 IHNob3VsZCB0byBiZSB1cGRhdGVkIHRvIG1hdGNoIHRoZSBjdXJyZW50IGNvbnRlbnQgb2YgdGhl IGRvY3VtZW50LiBJIHN1Z2dlc3QganVzdCBjb3B5aW5nIHRoZSA0dGggYW5kIDV0aCBwYXJhZ3Jh cGhzIGZyb20gdGhlIEludHJvZHVjdGlvbiBzZWN0aW9uLjxiciBjbGFzcz0iIj4NCjxiciBjbGFz cz0iIj4NCjwvYmxvY2txdW90ZT4NCjxkaXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+ DQo8ZGl2IGNsYXNzPSIiPlRoYW5rcyBmb3IgdGhlIHN1Z2dlc3Rpb24uIFRoZSBuZXcgdmVyc2lv biBpcyB1cGRhdGVkIGFjY29yZGluZ2x5LjwvZGl2Pg0KPGJyIGNsYXNzPSIiPg0KPGJsb2NrcXVv dGUgdHlwZT0iY2l0ZSIgY2xhc3M9IiI+LSBnZW5lcmFsIGNvbW1lbnQ6IGNsYXNzZXMgb2YgKERl dE5ldCkgU2VydmljZTxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NClRoZSBkb2N1bWVudCBz dGF0ZXMgdGhhdCBpdCB1c2VzIHRlcm1pbm9sb2d5IGZyb20gUkZDODY1NSwgaXQgYWxzbyB1c2Vz IHRoZSB0ZXJtcyAmcXVvdDtDbGFzc2VzIG9mIERldE5ldCBTZXJ2aWNlJnF1b3Q7LCAmcXVvdDsg RGV0TmV0IENsYXNzZXMgb2YgU2VydmljZSZxdW90OyZuYnNwOyBhbmQgb3RoZXIgc2ltaWxhciBm b3JtcyBvZiB0aGVzZSB0ZXJtcy4mbmJzcDsgUkZDODY1NSB1c2VzIHRoaXMgdGVybSBpbiBvbmUg cGxhY2UgJnF1b3Q7IENsYXNzLW9mLVNlcnZpY2Ugc2NoZW1lcyAoZS5nLiwgRGlmZlNlcnYpJnF1 b3Q7DQogYW5kIGFsc28gbWVudGlvbnMgJnF1b3Q7Y2xhc3NlcyBvZiBEZXROZXQgZmxvd3MmcXVv dDsmbmJzcDsgaW4gdHdvIHBsYWNlcy4mbmJzcDsgVGhlIGRhdGEgcGxhbmUgZG9jdW1lbnRzIFtS RkM4OTY0XSBhbmQgW1JGQzg5MzRdIGFsc28gZGVzY3JpYmUgQ2xhc3Mgb2YgU2VydmljZSBpbiB0 aGUgY29udGV4dCBvZiBEaWZmU2VydiAoYW5kIG5vdCBEZXROZXQpLjxiciBjbGFzcz0iIj4NCjxi ciBjbGFzcz0iIj4NCkluIHJlYWRpbmcgdGhlIGRvY3VtZW50IGl0IHNlZW1zIHRoYXQgaXQgaXMg bm90IHVzaW5nIENvUyBpbiB0aGUgdHlwaWNhbCB3YXkgdXNlZCBpbiB0aGUgSUVURiZuYnNwOyBv ciB0aGUgb3RoZXIgRGV0TmV0IFJGQ3MsIGFuZCBpcyBpbnRyb2R1Y2luZyBhIG5ldyBkZWZpbml0 aW9uIGZvciBDb1MuICZuYnNwOyBJSSB0aGluayBoYXZpbmcgYSBkb2N1bWVudCBzcGVjaWZpYyBk ZWZpbml0aW9uIG9mIENvUyBwcm9iYWJseSBpc24ndCBoZWxwZnVsLiBJdCB3b3VsZCBiZQ0KIGJl c3QgdG8gdXNlIGV4aXN0aW5nIERldE5ldCB0ZXJtaW5vbG9neS4mbmJzcDsgSSBzdXNwZWN0Jm5i c3A7ICZxdW90O2Zsb3cgYWdncmVnYXRlJnF1b3Q7IGlzIHRoZSBjbG9zZXN0LCBidXQgcGVyaGFw cyBJJ20gbWlzc2luZyBzb21ldGhpbmcuPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KPC9i bG9ja3F1b3RlPg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxkaXYgY2xh c3M9IiI+VGhhbmtzIGZvciB5b3VyIGNvbW1lbnQuIEluIHRoZSBuZXcgdmVyc2lvbiwgd2UgdXNl ZCB0aGUgdGVybSDigJxhZ2dyZWdhdGXigJ0gaW5zdGVhZCBvZiBjbGFzcyBpbiBzZWN0aW9uIDMu MSBhbmQgY2xhc3NlcyBvZiBEZXROZXQgZmxvd3MuIFNlY3Rpb24gNi40IHVzZXMgdGhlIHRlcm0g 4oCcdHJhZmZpYyBjbGFzc+KAnSBhcyBpbiBJRUVFODAyLjFRLiZuYnNwOzwvZGl2Pg0KPGRpdiBj bGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiIGNs YXNzPSIiPi0gYWxpZ25tZW50IHdpdGggdGhlIERhdGEgUGxhbmUgUkZDczxiciBjbGFzcz0iIj4N CjxiciBjbGFzcz0iIj4NClRoZSBsaXN0cyBpbiBzZWN0aW9uIDMuMSBhbmQgaW4gc2VjdGlvbiA2 LjQgYXJlIG5vdCB3ZWxsIGFsaWduZWQgd2l0aCB0aGUgRGF0YSBwbGFuZSBkb2N1bWVudHMuJm5i c3A7IFRoZXNlIHNlY3Rpb25zIHNob3VsZCBiZSBhbGlnbmVkIHdpdGggdGhlIHByb3Zpc2lvbmlu ZyBvZiBJUCBmbG93cywgYXMgc3VtbWFyaXplZCBpbiBTZWN0aW9uJm5ic3A7IDYgb2YgUkZDODkz OSwgYW5kIE1QTFMgYXMgc3VtbWFyaXplZCBpbiBTZWN0aW9uIDUgb2YgUkZDODk2NC48YnIgY2xh c3M9IiI+DQo8YnIgY2xhc3M9IiI+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2IGNsYXNzPSIiPjxmb250 IGZhY2U9IkhlbHZldGljYSBOZXVlIiBjbGFzcz0iIj5UaGFua3MgZm9yIHlvdXIgY29tbWVudC4g U2VjdGlvbiAzLjEgaXMgYWJvdXQgZmxvdyBhZG1pc3Npb24gcGFyYWRpZ20gYW5kIG5vdCAmcXVv dDtmbG93IGNyZWF0aW9u4oCdLiBUaGUgdGl0bGUgaXMgY2hhbmdlZCB0byZuYnNwO+KAnGZsb3cg YWRtaXNzaW9u4oCdIHRvIGF2b2lkIHBvc3NpYmxlIGNvbmZ1c2lvbi4gSW4gZmFjdCwgdGhpcyBz ZWN0aW9uIGlzIGFsaWduZWQgd2l0aCB0aGUNCiBEYXRhUGxhbmUgUkZDcyBhbmQgaW4gcGFyYWxs ZWwgd2l0aCBwcm92aXNpb25pbmcgb2YgSVAgZmxvd3MuIFdlIG1vZGlmaWVkIHNlY3Rpb24gNi40 IHRvIGZvbGxvdyZuYnNwO3RoZSBEYXRhIHBsYW5lIGRvY3VtZW50cy48L2ZvbnQ+PC9kaXY+DQo8 ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGJyIGNsYXNzPSIiPg0KPGJsb2Nr cXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9IiI+LSBtaW5vciBjb21tZW50IHNlcnZpY2UgdnMgZnJh bWUgcHJlZW1wdGlvbjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NClNlY3Rpb24gMy4xIHVz ZXMgdGhlIHRlcm0gJnF1b3Q7dW4tcHJvdmlzaW9uaW5nJnF1b3Q7LCBJIHJlYWQgdGhlIHRleHQg YXMgbWVhbmluZyAmcXVvdDtzZXJ2aWNlIHByZWVtcHRpb24mcXVvdDsuPGJyIGNsYXNzPSIiPg0K PGJyIGNsYXNzPSIiPg0KSW4gdGhlIHJlc3Qgb2YgdGhlIGRvY3VtZW50IHByZWVtcHRpb24gcmVm ZXJzIHRvICZxdW90O2ZyYW1lIHByZWVtcHRpb24mcXVvdDsgYW5kIHRoaXMgc2hvdWxkIGJlIGNs YXJpZmllZC48YnIgY2xhc3M9IiI+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2IGNsYXNzPSIiPjxiciBj bGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaGFua3MgZm9yIHRoZSBzdWdnZXN0aW9u LiBJbiB0aGUgbmV3IHZlcnNpb24sIHdlIHVzZSB0aGUgdGVybSDigJxzZXJ2aWNlIHByZWVtcHRp b27igJ0gYW5kIHVwZGF0ZSB0aGUgdGVybSDigJxwcmVlbXB0aW9u4oCdIHRvIOKAnGZyYW1lIHBy ZWVtcHRpb27igJ0uPC9kaXY+DQo8YnIgY2xhc3M9IiI+DQo8YmxvY2txdW90ZSB0eXBlPSJjaXRl IiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQotIGlkIG5pdHMsIHBsZWFzZSBlbnN1cmUgdGhlIGRv Y3VtZW50IHBhc3NlcyBpZG5pdHMgd2l0aG91dCBlcnJvcnMsIHNlZTxiciBjbGFzcz0iIj4NCjxi ciBjbGFzcz0iIj4NCjxhIGhyZWY9Imh0dHBzOi8vd3d3Ni5pZXRmLm9yZy90b29scy9pZG5pdHM/ dXJsPWh0dHBzOi8vd3d3LmlldGYub3JnL2FyY2hpdmUvaWQvZHJhZnQtaWV0Zi1kZXRuZXQtYm91 bmRlZC1sYXRlbmN5LTAyLnR4dCIgY2xhc3M9IiI+aHR0cHM6Ly93d3c2LmlldGYub3JnL3Rvb2xz L2lkbml0cz91cmw9aHR0cHM6Ly93d3cuaWV0Zi5vcmcvYXJjaGl2ZS9pZC9kcmFmdC1pZXRmLWRl dG5ldC1ib3VuZGVkLWxhdGVuY3ktMDIudHh0PC9hPjwvYmxvY2txdW90ZT4NCjxiciBjbGFzcz0i Ij4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaGFua3MgZm9yIHlvdXIgY29tbWVudCBhYm91dCBp ZG5pdHMuIFdlIGFkZGVkIHRoZSBtaXNzaW5nIHNlY3Rpb25zLCBpLmUuLCDigJxzZWN1cml0eSBj b25zaWRlcmF0aW9uc+KAnSBhbmQg4oCcSUFOQSBjb25zaWRlcmF0aW9uc+KAnSB0byB0aGUgZHJh ZnQuIEFsc28gcmVzb2x2ZWQgb3RoZXIgY29tbWVudHMgYnkgaWRuaXRzLjwvZGl2Pg0KPC9kaXY+ DQo8ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5CZXN0 LDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5FaHNhbjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj48YnIgY2xh c3M9IiI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2 IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9 IiI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+LS08YnIgY2xhc3M9IiI+DQpFaHNhbiBNb2hhbW1h ZHBvdXI8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+UGhEIENhbmRpZGF0ZSBhdCBTd2lzcyBGZWRlcmFs IEluc3RpdHV0ZSBvZiBUZWNobm9sb2d5IChFUEZMKTwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5FUEZM IEVESUMgUGhEIHN0dWRlbnQgcmVwcmVzZW50YXRpdmUmbmJzcDs8YnIgY2xhc3M9IiI+DQpJQyBJ SU5GQ09NLCBMQ0EyLCBJTkYgMDExLCBTdGF0aW9uIDE0LCZuYnNwOzEwMTUgTGF1c2FubmUsIFN3 aXR6ZXJsYW5kPGJyIGNsYXNzPSIiPg0KPGEgaHJlZj0iaHR0cHM6Ly9wZW9wbGUuZXBmbC5jaC9l aHNhbi5tb2hhbW1hZHBvdXI/bGFuZz1lbiIgY2xhc3M9IiI+aHR0cHM6Ly9wZW9wbGUuZXBmbC5j aC9laHNhbi5tb2hhbW1hZHBvdXI8L2E+PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPg0KPGRpdiBkaXI9 ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAs IDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRl bnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQt c3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3Jh dGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3Bh Y2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0i YXV0byIgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwg MCk7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVu dDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1z cGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0 aW9uOiBub25lOyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFj ZTsgbGluZS1icmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJh dXRvIiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAw KTsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50 OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNw YWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRp b246IG5vbmU7IHdvcmQtd3JhcDogYnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNl OyBsaW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1 dG8iIHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDAp OyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6 IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQtc3Bh Y2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlv bjogbm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7 IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0 byIgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDog MHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFj aW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9u OiBub25lOyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsg bGluZS1icmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRv IiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsg bGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAw cHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNp bmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246 IG5vbmU7IHdvcmQtd3JhcDogYnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBs aW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8i IHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBs ZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBw eDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQtc3BhY2lu ZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlvbjog bm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxp bmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIg c3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGxl dHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4 OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5n OiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBu b25lOyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGlu ZS1icmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBz dHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0 dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7 IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6 IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5v bmU7IHdvcmQtd3JhcDogYnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5l LWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBzdHlsZT0iY2FyZXQt Y29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEhl bHZldGljYTsgZm9udC1zaXplOiAxMnB4OyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFu dC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGxldHRlci1zcGFjaW5nOiBub3Jt YWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTog bm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4 dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyI+DQo8YnIgY2xhc3M9 IiI+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rp dj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj48YnIgY2xhc3M9IiI+DQo8 YmxvY2txdW90ZSB0eXBlPSJjaXRlIiBjbGFzcz0iIj4NCjxkaXYgY2xhc3M9IiI+T24gMjEgRmVi IDIwMjEsIGF0IDIzOjE1LCBMb3UgQmVyZ2VyICZsdDs8YSBocmVmPSJtYWlsdG86bGJlcmdlckBs YWJuLm5ldCIgY2xhc3M9IiI+bGJlcmdlckBsYWJuLm5ldDwvYT4mZ3Q7IHdyb3RlOjwvZGl2Pg0K PGJyIGNsYXNzPSJBcHBsZS1pbnRlcmNoYW5nZS1uZXdsaW5lIj4NCjxkaXYgY2xhc3M9IiI+DQo8 ZGl2IGNsYXNzPSIiPkFsbCw8YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGUgV0cgbGFz dCBjYWxsIGlzIGNvbXBsZXRlLjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCkF1dGhvcnMs PGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KUGxlYXNlIGJyaW5nIGFueSByZXNvbHV0aW9u cyB0byBjb21tZW50cyByZWNlaXZlZCB0byB0aGUgbGlzdC4gT25jZSBhbGwgY29tbWVudHMgYXJl IGFkZHJlc3NlZCBwbGVhc2UgdXBkYXRlIHRoZSBkcmFmdCBhbmQgbGV0IHRoZSBXRyBrbm93IHRo YXQgaXQgaXMgcmVhZHkgZm9yIHN1Ym1pc3Npb24gdG8gdGhlIElFU0cgZm9yIHB1YmxpY2F0aW9u cy48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGFuayB5b3UhPGJyIGNsYXNzPSIiPg0K PGJyIGNsYXNzPSIiPg0KTG91PGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KT24gMi81LzIw MjEgODozMCBBTSwgTG91IEJlcmdlciB3cm90ZTo8YnIgY2xhc3M9IiI+DQo8YmxvY2txdW90ZSB0 eXBlPSJjaXRlIiBjbGFzcz0iIj5BbGwsPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KVGhp cyBzdGFydHMgd29ya2luZyBncm91cCBsYXN0IGNhbGwgb24gZHJhZnQtaWV0Zi1kZXRuZXQtYm91 bmRlZC1sYXRlbmN5LTAyPGJyIGNsYXNzPSIiPg0KPGEgaHJlZj0iaHR0cHM6Ly9kYXRhdHJhY2tl ci5pZXRmLm9yZy9kb2MvZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LyIgY2xhc3M9 IiI+aHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtaWV0Zi1kZXRuZXQtYm91 bmRlZC1sYXRlbmN5LzwvYT48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGUgd29ya2lu ZyBncm91cCBsYXN0IGNhbGwgZW5kcyBvbiBGZWJydWFyeSAxOXRoLjxiciBjbGFzcz0iIj4NClBs ZWFzZSBzZW5kIHlvdXIgY29tbWVudHMgdG8gdGhlIHdvcmtpbmcgZ3JvdXAgbWFpbGluZyBsaXN0 LjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NClBsZWFzZSBub3RlLCB0aGVyZSB3YXMgYW4g SVBSIGRpc2Nsb3N1cmUgYWdhaW5zdCB0aGlzIGRvY3VtZW50PGJyIGNsYXNzPSIiPg0Kc3VibWl0 dGVkIG9uIEphbnVhcnkgMTQsIDIwMjEsIHNlZSBodHRwczovL2RhdGF0cmFja2VyLmlldGYub3Jn L2lwci80NTgxLzxiciBjbGFzcz0iIj4NCldoaWxlIHRoaXMgZGlzY2xvc3VyZSBjYW1lIHZlcnkg bGF0ZSBpbiB0aGUgZG9jdW1lbnQgbGlmZSBjeWNsZSw8YnIgY2xhc3M9IiI+DQppdCBhcHBlYXJz IHRoZSBmaWxpbmcgd2FzIGFsc28gcmVsYXRpdmVseSByZWNlbnQgKDIwMTktMTAtMTYpIGFuZDxi ciBjbGFzcz0iIj4NCmFmdGVyIHRoZSBwcmUgYWRvcHRpb24gSVAgY2FsbDo8YnIgY2xhc3M9IiI+ DQpodHRwczovL21haWxhcmNoaXZlLmlldGYub3JnL2FyY2gvbXNnL2RldG5ldC8xbXdZZGFkRHpU THVkY3VINVQ1TzdSX0tsRHM8YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpQb3NpdGl2ZSBj b21tZW50cywgZS5nLiwgJnF1b3Q7SSd2ZSByZXZpZXdlZCB0aGlzIGRvY3VtZW50PGJyIGNsYXNz PSIiPg0KYW5kIGJlbGlldmUgaXQgaXMgcmVhZHkgZm9yIHB1YmxpY2F0aW9uJnF1b3Q7LCBhcmUg d2VsY29tZSE8YnIgY2xhc3M9IiI+DQpUaGlzIGlzIHVzZWZ1bCBhbmQgaW1wb3J0YW50LCBldmVu IGZyb20gYXV0aG9ycy48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGFuayB5b3UsPGJy IGNsYXNzPSIiPg0KTG91IChEZXROZXQgQ28tQ2hhaXIgJmFtcDsgZG9jIFNoZXBoZXJkKTxiciBj bGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyIGNsYXNzPSIi Pg0KZGV0bmV0IG1haWxpbmcgbGlzdDxiciBjbGFzcz0iIj4NCmRldG5ldEBpZXRmLm9yZzxiciBj bGFzcz0iIj4NCmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vZGV0bmV0PGJy IGNsYXNzPSIiPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4N CjwvZGl2Pg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4N Cg== --_000_8608941FAD44409883DF416FBD2DFEA3epflch_-- From nobody Mon Mar 22 08:56:52 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE1D93A0A7D; Mon, 22 Mar 2021 08:56:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=labn.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 303LnZH1WMwG; Mon, 22 Mar 2021 08:56:45 -0700 (PDT) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2106.outbound.protection.outlook.com [40.107.93.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 591BF3A0A79; Mon, 22 Mar 2021 08:56:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DWbMyIoDpX7dAz9ed3p0PMP89Dn9Ys4i1lldzgdUgRKj1/jdnfiTsrGpw5IgOOoK7koAjApT/q8iuhv+IjrW/larkeXTmTE/fRWNvg5hPrigtJu6Llz8mhmnQ2KxVaTCqLxWZmNXy6boWWOBPobqF/AxgbO/1zZ3Ujxy1K9+QtrmxiH0TRHVZAcjA8P+ptSpA66oUMFNtljOkUNxuFfOOhetmuR/4lhNbAIRnOlzGAEJ0tVfk5VgWuL/v0d/v0/LOKYXSEQK3g5EN+1S1heNZ/nXToQg3pbz8bFeONwK8iaGxeV2D8QPtSo2XT3SwMtd0XYfiuCmcwtMlf9j4ObRxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8+mt9k7NrIBcXC/ObcRRTumobg58E36yHdMg/Dtr02g=; b=RcZ7tKneo5B7zB1PKIxZhC6sJGvDtPSY6x8TblaN+php/wMK1uTv/UPS2YGMMXhFoOIdRO8TE5Z0O/wueVT5lryfBOaj53TAvDOOGeeBXNhaV1dgwvF4Or13/z9IwlEZXqiMqYq0G+S6UAMUQIhiEPuG2vKst0LbrPJjIP3DcKQQ4nSSBMHns1z4A0oxexqIJnZSBRb5XKWmri35LyxS7t7FULb6zN3gBNs68KyII4BeeqIBYct+sNZ4JUgJR0eamXgaHoZ+htPMPyQl+BlHKFY0+887U6cyqV2uldvZaiykcRXfrQAsLOHxmZXw7e/kP5emTAYhfXFFrWxFVjiQgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=labn.net; dmarc=pass action=none header.from=labn.net; dkim=pass header.d=labn.net; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=labn.onmicrosoft.com; s=selector2-labn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8+mt9k7NrIBcXC/ObcRRTumobg58E36yHdMg/Dtr02g=; b=eWPft9IXOITh4nz3P/1BG9DIbsHHY0LcgNJcv84rG+gBnyxCevsWEMWlNPCKDDZz3xDX37dYMDTJc+U3iItr5CfQjiGr7OA8CQ4J+ki7OkLGMZNAsbslxkboq42td4rQMrHdfzbfjW38q04WzeiXLRD3qUd8MWihaRYWEAd2VDc= Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=labn.net; Received: from BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) by MN2PR14MB3277.namprd14.prod.outlook.com (2603:10b6:208:1b2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18; Mon, 22 Mar 2021 15:56:42 +0000 Received: from BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b]) by BL0PR14MB3779.namprd14.prod.outlook.com ([fe80::1ff:75f:e082:452b%8]) with mapi id 15.20.3955.027; Mon, 22 Mar 2021 15:56:42 +0000 To: Mohammadpour Ehsan Cc: DetNet WG , "draft-ietf-detnet-bounded-latency@ietf.org" References: <371c2363-4f15-b32f-4c96-483a31ae1c77@labn.net> <8608941F-AD44-4098-83DF-416FBD2DFEA3@epfl.ch> From: Lou Berger Message-ID: Date: Mon, 22 Mar 2021 11:56:40 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 In-Reply-To: <8608941F-AD44-4098-83DF-416FBD2DFEA3@epfl.ch> Content-Type: multipart/alternative; boundary="------------01AB6B355C86E124CE178BB7" Content-Language: en-US X-Originating-IP: [100.15.108.238] X-ClientProxiedBy: MN2PR11CA0002.namprd11.prod.outlook.com (2603:10b6:208:23b::7) To BL0PR14MB3779.namprd14.prod.outlook.com (2603:10b6:208:1c7::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [127.0.0.1] (100.15.108.238) by MN2PR11CA0002.namprd11.prod.outlook.com (2603:10b6:208:23b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Mon, 22 Mar 2021 15:56:42 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 30f6e303-baee-469c-bbee-08d8ed4b161d X-MS-TrafficTypeDiagnostic: MN2PR14MB3277: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mhaS3nJZvw23nsB5QIfFTTqPLENAaMsol4odkuS9OWQbvNSxR51GZ337xywNe9D8XKGTNBBjO2xKpZOoETih4JSf3wiSTcmt+mlOHtfOY+JBh+DvVA91JlQ9PiCHByehnib84TVSqnPYhBZI+lDuw4CHxSb6DvwcVbHTP+nb/u/Y1ittkz+XQu6N799yzEyWCyadDcWGGXYpQqzUq+DH+NfR8AAZr2aPo/elWwzpIUFA0fIfQdGZOPfX0RpT/rpeNdrz2rQvF68t5D4PaBo1oml8WPVd9zbqoBJXi5Z5eB5jHhXDnP7lHt3hKWstHrxfCJWG2Y0e0+R+fNGLdAs1StzOBF0ld5Ww4b/yipa0NZ75DNdFfFc9Go++GSJgCisJzFc3550JhRHQhppd4XJTDpdpnIZdDtbBZV0Maqx8cyE8AzGI7b663pNC/3bErR27v0GlH2ODo2gOctIWwXvuyjoZgu7dRPvC21fYIdxol4ouLl8H9+EOnRs75wUr4n/lT0uTxQS5dzGbPeqaCWgnOHD2Cw+Zr1/g6H62GNWEP7+VI+3ieVABIGocgAzhsTSCzasPSP1qhxOUL6DJGE531G45tjVx3iSqrLzwYbG4oV8ke5cwvqfrxCXaLgYPGkBJxlH+Mdn7kRmZnqvSqM9dndvj4j3rrj0XfQ6aPTocQVq6M9uXzlFZbCjxMz69k7NXkSEvtc7zsqvqnE2NYYijKKiUFB3xpp8mLxx3Yc4CGfcmag4VQJznEjQCao8uLQDRq9ZpSc2Mr0KDiAgwOGzljpcRtPogodhON/d9Q0RoMbs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR14MB3779.namprd14.prod.outlook.com; PTR:; CAT:NONE; SFS:(346002)(366004)(396003)(136003)(39830400003)(376002)(8936002)(7246003)(16526019)(83380400001)(31686004)(54906003)(7126003)(956004)(4326008)(30864003)(8676002)(6916009)(36756003)(186003)(966005)(2616005)(26005)(16576012)(316002)(166002)(53546011)(2906002)(66476007)(66556008)(5660300002)(33964004)(66946007)(31696002)(52116002)(86362001)(38100700001)(4001150100001)(478600001)(45640500001)(6486002)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?Q1o2QkNTS1hXM0ZZMFBSeVBTbHk2SVJrQVk0OG1ybWJjbFZBbXNWR3B5cm5n?= =?utf-8?B?SlJZQlNmOGVhSEl3Z29FWG5qQnJYWHZIVGQzc0syQ2FlbU9VZVY4VE4xaU9F?= =?utf-8?B?QVpmbU9lYVZvcWdmQVNzYmFOakhyeWVienVwTkhzTXNXSzlCTmdiOTE4SU9n?= =?utf-8?B?TEF0dzRlNkVQRjEvc3o1YTZjTWFCWWNzNDBrenJ6bitDVkNsMTh1MWlhRWQ3?= =?utf-8?B?M0YycWFiS1JaSmcxeWt5ZlF6bjVrQVdVejJtNXdZeFVxckRBQlRrQk5VMnBW?= =?utf-8?B?TmdheEpKN09nLzI3SE9mTC9tc1VTbVBpZUgyUmVXNzdzVVBtVTcwV3BrS3lF?= =?utf-8?B?ak9sY25hQm9YaVdoQ0tMNndEaUZzMVBRRFFGQ3E4SGNFVlozOGZmME9TQ0ds?= =?utf-8?B?aDVzbHc0bmZzTHJmaTIwemZKVWRYRTdiL0MvdXg1M0lWSU1vU3M4d09Oazk5?= =?utf-8?B?Smg1NEdEdlEzVHVrVXVYcXNGTUsyQ05tMnd2dTlnNGd2MG1PbXhnazgvN2tw?= =?utf-8?B?K2hJZGdGVTVWWkFUV0VxSW9qRUdqK3dGVWo1VExFVXFxK1NDOWlvTVo3UjhE?= =?utf-8?B?aUk1Y282ejJxcVlyQzl6Y0hwbUloZ3JkaW9nY1J0REtpMGw3UU9GQXF2RmU0?= =?utf-8?B?NjhBZnFZSlBWMEszTThTZGJBVTZpRXc2S2FKQ3JwUzVGQkp3RVlrOHVVd0tL?= =?utf-8?B?aEdTR2hFRHJMTTJ1K2QxNmlMakNkeFRjVCtUMlNkL1JWUVB3a1VkNGsxYk5H?= =?utf-8?B?V04zTTJQZW9jSS8yZ2VJQTJzcWdqblAzKzMyMVFNR3BtL3l2VFFDOEN5S0Fw?= =?utf-8?B?OHZ2NUZNb2pHSStzRTV2SmVmUzlQZmhUaHZzS3dUWkE0akQzYy9zQXh6NkRM?= =?utf-8?B?VUd5K0w5aXltZjhLeUpLN2xoaUg5K284OUpId0ZPRE1DWXA2cmNVeWtraG9E?= =?utf-8?B?RlhvbVp6d3o0ekYwbnJzRlV1U2UwTVF4R3lNUFNiT3RtbFlhTzVQb0tDeEgv?= =?utf-8?B?VFd1T0QxRXlrUzhybHlzckZBd1NiaFV6UmQxYjlkRVpiUWFPVkd3aXlDbW9n?= =?utf-8?B?bDVjNnFLSDJRNkpFZTdtdVdSbkpPc2t1YWp2QjV6MFllUmZ5WjdHVmIvU3Ez?= =?utf-8?B?Ykt1dVMxK3QranlqR1NVMVp1OHljUDVTTkN4UytueDFNdDVpTnZnZkl4MFBY?= =?utf-8?B?Q01zeVpITjBQdkJ5TEdSMGpmS0FPdXNsc1NxcXJTaysySVkzdkV3ZFB0ajB4?= =?utf-8?B?WmNXZlYwZlpuSVg1dUdmUlNGYU5tK1FzQ3R1UUZ0Z3p0eTlFWENhQUpsSkh6?= =?utf-8?B?YVJDR1ppQ2FIR0pYdDRQaHNYQ29qczlWb2szOS96SDJRV2VaeTBzd0Fidi9O?= =?utf-8?B?amhtc3M4QXIzdGJRbnFOZklqZmxuY2NPSDNMUVptcDdmOERLdzNrOVZ0cCs2?= =?utf-8?B?SUtiOUlHVTV0NGpQYTJJazZGeXNpbTN1K1RJTnJGUlRjVTgrS3dtZW9HRnVz?= =?utf-8?B?dEN6bzRrMmo1OFlFVy9VUWVBQmhSTW5VT0VmNzZ4dld3eXU1T2xXeXJNZHN1?= =?utf-8?B?cmhHT2RHMHdLZ3Z5RGEvMmhxYXBEYzZNYVliRjlwL0w3M0pqc3RLRjE4Z1Js?= =?utf-8?B?eURFQ1k5aFNISHZhVFBOZ0hNVGlqMXNGTmVqdjhRYklvaEhrOWFlWmFHNkpj?= =?utf-8?B?ZUFxQUxPMzdQL2VZU1dhQUpUM0FLY0ZxeTFRa0c0RkNPTXZxb0dwOXR0OU5B?= =?utf-8?Q?eaG/DPofpuBfhEQ6FAQt4MNm9o04hrEw3OBs1Vv?= X-OriginatorOrg: labn.net X-MS-Exchange-CrossTenant-Network-Message-Id: 30f6e303-baee-469c-bbee-08d8ed4b161d X-MS-Exchange-CrossTenant-AuthSource: BL0PR14MB3779.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Mar 2021 15:56:42.4831 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: eb60ac54-2184-4344-9b60-40c8b2b72561 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hHicl1xrLNs2KGFHFov0Rfg0TLJbuyhkawG63GN4YFV54zrPlFi1NYZwGMvF9jGV/ROVM8iwG1fx8ZNFZCKySQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR14MB3277 Archived-At: Subject: Re: [Detnet] WG Last Call: draft-ietf-detnet-bounded-latency-02 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 15:56:51 -0000 --------------01AB6B355C86E124CE178BB7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Ehsan Thank you for the update. I did a quick skim and it looks much improved. I found one bit of new language confusing:    Section 6.4    In the considered queuing model, we considered the four traffic    classes (Definition 3.268 of [IEEE8021Q]): control-data traffic    (CDT), class A, class B, and best effort (BE) in decreasing order of    priority. Flows of classes A and B are together referred as AVB    flows. This model is a subset of Time-Sensitive Networking as    described next.    ....    Then,    the input flows are identified using the information in (Section 5.1    of [RFC8939]). Then they are aggregated into eight macro flows based    on their traffic classes. We refer to each macro flow as a class. So in the first paragraph, you say "traffic classes" is defined by 802.1Q. in the second, you say "their traffic classes" is defined based on [RFC8939]. I believe these are inconsistent definitions. I think you are trying to say that     Then they are aggregated into eight macro flows based on their *DetNet flow aggregate.* This is assuming that the node isn't doing some sort of independent mapping of micro-flows to macro-flows. Please let me know if I'm missing something. Lou On 3/22/2021 11:33 AM, Mohammadpour Ehsan wrote: > Dear WG, > > We submitted a new version of the Bounded Latency draft that addresses > the comments received in the list > (https://datatracker.ietf.org/doc/html/draft-ietf-detnet-bounded-latency-04 > ); > it is now ready for submission to the IESG for publication. > > Thanks Lou for his comments. In the new version, we address the > comments as follows: > >> - Abstract >> I think the abstract should to be updated to match the current >> content of the document. I suggest just copying the 4th and 5th >> paragraphs from the Introduction section. >> > > Thanks for the suggestion. The new version is updated accordingly. > >> - general comment: classes of (DetNet) Service >> >> The document states that it uses terminology from RFC8655, it also >> uses the terms "Classes of DetNet Service", " DetNet Classes of >> Service" and other similar forms of these terms. RFC8655 uses this >> term in one place " Class-of-Service schemes (e.g., DiffServ)" and >> also mentions "classes of DetNet flows" in two places. The data >> plane documents [RFC8964] and [RFC8934] also describe Class of >> Service in the context of DiffServ (and not DetNet). >> >> In reading the document it seems that it is not using CoS in the >> typical way used in the IETF or the other DetNet RFCs, and is >> introducing a new definition for CoS. II think having a document >> specific definition of CoS probably isn't helpful. It would be best >> to use existing DetNet terminology. I suspect "flow aggregate" is >> the closest, but perhaps I'm missing something. >> > > Thanks for your comment. In the new version, we used the term > “aggregate” instead of class in section 3.1 and classes of DetNet > flows. Section 6.4 uses the term “traffic class” as in IEEE802.1Q. > >> - alignment with the Data Plane RFCs >> >> The lists in section 3.1 and in section 6.4 are not well aligned with >> the Data plane documents. These sections should be aligned with the >> provisioning of IP flows, as summarized in Section 6 of RFC8939, and >> MPLS as summarized in Section 5 of RFC8964. >> > Thanks for your comment. Section 3.1 is about flow admission paradigm > and not "flow creation”. The title is changed to “flow admission” to > avoid possible confusion. In fact, this section is aligned with the > DataPlane RFCs and in parallel with provisioning of IP flows. We > modified section 6.4 to follow the Data plane documents. > > >> - minor comment service vs frame preemption >> >> Section 3.1 uses the term "un-provisioning", I read the text as >> meaning "service preemption". >> >> In the rest of the document preemption refers to "frame preemption" >> and this should be clarified. > > Thanks for the suggestion. In the new version, we use the term > “service preemption” and update the term “preemption” to “frame > preemption”. > >> >> - id nits, please ensure the document passes idnits without errors, see >> >> https://www6.ietf.org/tools/idnits?url=https://www.ietf.org/archive/id/draft-ietf-detnet-bounded-latency-02.txt >> > > Thanks for your comment about idnits. We added the missing sections, > i.e., “security considerations” and “IANA considerations” to the > draft. Also resolved other comments by idnits. > > Best, > Ehsan > > > > > -- > Ehsan Mohammadpour > PhD Candidate at Swiss Federal Institute of Technology (EPFL) > EPFL EDIC PhD student representative > IC IINFCOM, LCA2, INF 011, Station 14, 1015 Lausanne, Switzerland > https://people.epfl.ch/ehsan.mohammadpour > > > >> On 21 Feb 2021, at 23:15, Lou Berger > > wrote: >> >> All, >> >> The WG last call is complete. >> >> Authors, >> >> Please bring any resolutions to comments received to the list. Once >> all comments are addressed please update the draft and let the WG >> know that it is ready for submission to the IESG for publications. >> >> Thank you! >> >> Lou >> >> On 2/5/2021 8:30 AM, Lou Berger wrote: >>> All, >>> >>> This starts working group last call on >>> draft-ietf-detnet-bounded-latency-02 >>> https://datatracker.ietf.org/doc/draft-ietf-detnet-bounded-latency/ >>> >>> >>> The working group last call ends on February 19th. >>> Please send your comments to the working group mailing list. >>> >>> Please note, there was an IPR disclosure against this document >>> submitted on January 14, 2021, see >>> https://datatracker.ietf.org/ipr/4581/ >>> While this disclosure came very late in the document life cycle, >>> it appears the filing was also relatively recent (2019-10-16) and >>> after the pre adoption IP call: >>> https://mailarchive.ietf.org/arch/msg/detnet/1mwYdadDzTLudcuH5T5O7R_KlDs >>> >>> Positive comments, e.g., "I've reviewed this document >>> and believe it is ready for publication", are welcome! >>> This is useful and important, even from authors. >>> >>> Thank you, >>> Lou (DetNet Co-Chair & doc Shepherd) >>> >>> >>> >>> _______________________________________________ >>> detnet mailing list >>> detnet@ietf.org >>> https://www.ietf.org/mailman/listinfo/detnet > --------------01AB6B355C86E124CE178BB7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Ehsan

Thank you for the update. I did a quick skim and it looks much improved. I found one bit of new language confusing:

   Section 6.4

   In the considered queuing model, we considered the four traffic
   classes (Definition 3.268 of [IEEE8021Q]): control-data traffic
   (CDT), class A, class B, and best effort (BE) in decreasing order of
   priority. Flows of classes A and B are together referred as AVB
   flows. This model is a subset of Time-Sensitive Networking as
   described next.

   ....

   Then,
   the input flows are identified using the information in (Section 5.1
   of [RFC8939]). Then they are aggregated into eight macro flows based
   on their traffic classes. We refer to each macro flow as a class.

So in the first paragraph, you say "traffic classes" is defined by 802.1Q. in the second, you say "their traffic classes" is defined based on [RFC8939]. I believe these are inconsistent definitions. I think you are trying to say that

    Then they are aggregated into eight macro flows based on their *DetNet flow aggregate.*

This is assuming that the node isn't doing some sort of independent mapping of micro-flows to macro-flows.

Please let me know if I'm missing something.

Lou

On 3/22/2021 11:33 AM, Mohammadpour Ehsan wrote:

Dear WG,

We submitted a new version of the Bounded Latency draft that addresses the comments received in the list (https://datatracker.ietf.org/doc/html/draft-ietf-detnet-bounded-latency-04); it is now ready for submission to the IESG for publication.

Thanks Lou for his comments. In the new version, we address the comments as follows:

- Abstract
I think the abstract should to be updated to match the current content of the document. I suggest just copying the 4th and 5th paragraphs from the Introduction section.

Thanks for the suggestion. The new version is updated accordingly.

- general comment: classes of (DetNet) Service

The document states that it uses terminology from RFC8655, it also uses the terms "Classes of DetNet Service", " DetNet Classes of Service" and other similar forms of these terms. RFC8655 uses this term in one place " Class-of-Service schemes (e.g., DiffServ)" and also mentions "classes of DetNet flows" in two places. The data plane documents [RFC8964] and [RFC8934] also describe Class of Service in the context of DiffServ (and not DetNet).

In reading the document it seems that it is not using CoS in the typical way used in the IETF or the other DetNet RFCs, and is introducing a new definition for CoS. II think having a document specific definition of CoS probably isn't helpful. It would be best to use existing DetNet terminology. I suspect "flow aggregate" is the closest, but perhaps I'm missing something.

Thanks for your comment. In the new version, we used the term “aggregate” instead of class in section 3.1 and classes of DetNet flows. Section 6.4 uses the term “traffic class” as in IEEE802.1Q.

- alignment with the Data Plane RFCs

The lists in section 3.1 and in section 6.4 are not well aligned with the Data plane documents. These sections should be aligned with the provisioning of IP flows, as summarized in Section 6 of RFC8939, and MPLS as summarized in Section 5 of RFC8964.

Thanks for your comment. Section 3.1 is about flow admission paradigm and not "flow creation”. The title is changed to “flow admission” to avoid possible confusion. In fact, this section is aligned with the DataPlane RFCs and in parallel with provisioning of IP flows. We modified section 6.4 to follow the Data plane documents.

- minor comment service vs frame preemption

Section 3.1 uses the term "un-provisioning", I read the text as meaning "service preemption".

In the rest of the document preemption refers to "frame preemption" and this should be clarified.

Thanks for the suggestion. In the new version, we use the term “service preemption” and update the term “preemption” to “frame preemption”.

- id nits, please ensure the document passes idnits without errors, see

https://www6.ietf.org/tools/idnits?url=https://www.ietf.org/archive/id/draft-ietf-detnet-bounded-latency-02.txt

Thanks for your comment about idnits. We added the missing sections, i.e., “security considerations” and “IANA considerations” to the draft. Also resolved other comments by idnits.

Best,

Ehsan

--
Ehsan Mohammadpour

PhD Candidate at Swiss Federal Institute of Technology (EPFL)

EPFL EDIC PhD student representative
IC IINFCOM, LCA2, INF 011, Station 14, 1015 Lausanne, Switzerland
https://people.epfl.ch/ehsan.mohammadpour

On 21 Feb 2021, at 23:15, Lou Berger <lberger@labn.net> wrote:

All,

The WG last call is complete.

Authors,

Please bring any resolutions to comments received to the list. Once all comments are addressed please update the draft and let the WG know that it is ready for submission to the IESG for publications.

Thank you!

Lou

On 2/5/2021 8:30 AM, Lou Berger wrote:

All,

This starts working group last call on draft-ietf-detnet-bounded-latency-02
https://datatracker.ietf.org/doc/draft-ietf-detnet-bounded-latency/

The working group last call ends on February 19th.
Please send your comments to the working group mailing list.

Please note, there was an IPR disclosure against this document
submitted on January 14, 2021, see https://datatracker.ietf.org/ipr/4581/
While this disclosure came very late in the document life cycle,
it appears the filing was also relatively recent (2019-10-16) and
after the pre adoption IP call:
https://mailarchive.ietf.org/arch/msg/detnet/1mwYdadDzTLudcuH5T5O7R_KlDs

Positive comments, e.g., "I've reviewed this document
and believe it is ready for publication", are welcome!
This is useful and important, even from authors.

Thank you,
Lou (DetNet Co-Chair & doc Shepherd)

_______________________________________________
detnet mailing list
detnet@ietf.org
https://www.ietf.org/mailman/listinfo/detnet

--------------01AB6B355C86E124CE178BB7-- From nobody Thu Mar 25 00:56:17 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0D193A149F for ; Thu, 25 Mar 2021 00:56:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=epfl.ch Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zkjNzq91fL6N for ; Thu, 25 Mar 2021 00:56:10 -0700 (PDT) Received: from smtp4.epfl.ch (smtp4.epfl.ch [IPv6:2001:620:618:1e0:1:80b2:e059:1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 946B53A148C for ; Thu, 25 Mar 2021 00:56:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=epfl.ch; s=epfl; t=1616658962; h=From:To:CC:Date:Message-ID:Content-Type:MIME-Version; bh=0NjJ8ihhsV6QwCVm5dc+7mvIDo6AmSlmkAqECr9sFUI=; l=40858; b=XqEGWN/pcqZhRAIVGIUr15xldbJBLjjB/j9cxVheUQAPG/EOdNgc8U1kg0pWHIv84 K3PPTEb4svNFo8XueRStP1yNlWhVBkeiZdbfF8HA4KF2NKIg8U2uF8uW07gPr4cWo J/fNLE9SFAcWaWmmJ6qBLzwqKVJ5nyDg477B+Qisc= Received: (qmail 15170 invoked by uid 107); 25 Mar 2021 07:56:02 -0000 Received: from ax-snat-224-178.epfl.ch (HELO ewa07.intranet.epfl.ch) (192.168.224.178) (TLS, AES256-GCM-SHA384 cipher) by mail.epfl.ch (AngelmatoPhylax SMTP proxy) with ESMTPS; Thu, 25 Mar 2021 08:56:02 +0100 X-EPFL-Auth: D4nBpblrqeKuZDQlmV0P1cWf+UWsS76NmAFJmBYIW9BBUf5OxdA= Received: from ewa02.intranet.epfl.ch (128.178.224.159) by ewa07.intranet.epfl.ch (128.178.224.178) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Thu, 25 Mar 2021 08:56:01 +0100 Received: from ewa02.intranet.epfl.ch ([fe80::8939:e5e2:d561:d768]) by ewa02.intranet.epfl.ch ([fe80::8939:e5e2:d561:d768%4]) with mapi id 15.01.2106.013; Thu, 25 Mar 2021 08:56:01 +0100 From: Mohammadpour Ehsan To: Lou Berger CC: DetNet WG , "draft-ietf-detnet-bounded-latency@ietf.org" Thread-Topic: [Detnet] WG Last Call: draft-ietf-detnet-bounded-latency-02 Thread-Index: AQHW+8MLF9W102eQAk2Yhw/0iYu+aKpjNeAAgC0jTQCAAAZuAIAEMLOA Date: Thu, 25 Mar 2021 07:56:01 +0000 Message-ID: <6D2EFDE8-452C-40C7-B4AF-5300B995FE2B@epfl.ch> References: <371c2363-4f15-b32f-4c96-483a31ae1c77@labn.net> <8608941F-AD44-4098-83DF-416FBD2DFEA3@epfl.ch> In-Reply-To: Accept-Language: en-US, fr-CH Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.179.254.217] Content-Type: multipart/alternative; boundary="_000_6D2EFDE8452C40C7B4AF5300B995FE2Bepflch_" MIME-Version: 1.0 Archived-At: Subject: Re: [Detnet] WG Last Call: draft-ietf-detnet-bounded-latency-02 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2021 07:56:15 -0000 --_000_6D2EFDE8452C40C7B4AF5300B995FE2Bepflch_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGVsbG8gTG91LA0KDQpUaGFua3MgZm9yIHRoZSBjb21tZW50LCB5b3XigJlyZSByaWdodCEgSSB3 aWxsIHVzZSB0aGUgZm9sbG93aW5nIHNlbnRlbmNlOg0KDQoiVGhlbiB0aGV5IGFyZSBhZ2dyZWdh dGVkIGludG8gZWlnaHQgbWFjcm8gZmxvd3MgYmFzZWQgb24gdGhlaXIgRGV0TmV0IGZsb3cgYWdn cmVnYXRlLiINCg0KQmVzdCwNCkVoc2FuDQoNCg0KUC5TLiBUaGUgaW50ZW50aW9uIG9mICJUaGVu LCB0aGUgaW5wdXQgZmxvd3MgYXJlIGlkZW50aWZpZWQgdXNpbmcgdGhlIGluZm9ybWF0aW9uIGlu IChTZWN0aW9uIDUuMSBvZiBbUkZDODkzOV0p4oCdLCB3YXMgdG8gc2F5IHRoYXQgd2UgdXNlIFNl Y3Rpb24gNS4xIG9mIFJGQzg5MzkgdG8gaWRlbnRpZnkgdGhlIGluZGl2aWR1YWwgRGV0TmV0IGZs b3dzLg0KDQoNCg0KLS0NCkVoc2FuIE1vaGFtbWFkcG91cg0KUGhEIENhbmRpZGF0ZSBhdCBTd2lz cyBGZWRlcmFsIEluc3RpdHV0ZSBvZiBUZWNobm9sb2d5IChFUEZMKQ0KSUMgSUlORkNPTSwgTENB MiwgSU5GIDAxMSwgU3RhdGlvbiAxNCwgMTAxNSBMYXVzYW5uZSwgU3dpdHplcmxhbmQNCmh0dHBz Oi8vcGVvcGxlLmVwZmwuY2gvZWhzYW4ubW9oYW1tYWRwb3VyPGh0dHBzOi8vcGVvcGxlLmVwZmwu Y2gvZWhzYW4ubW9oYW1tYWRwb3VyP2xhbmc9ZW4+DQoNCk9uIDIyIE1hciAyMDIxLCBhdCAxNjo1 NiwgTG91IEJlcmdlciA8bGJlcmdlckBsYWJuLm5ldDxtYWlsdG86bGJlcmdlckBsYWJuLm5ldD4+ IHdyb3RlOg0KDQoNCkVoc2FuDQoNClRoYW5rIHlvdSBmb3IgdGhlIHVwZGF0ZS4gIEkgZGlkIGEg cXVpY2sgc2tpbSBhbmQgaXQgbG9va3MgbXVjaCBpbXByb3ZlZC4gIEkgZm91bmQgb25lIGJpdCBv ZiBuZXcgbGFuZ3VhZ2UgY29uZnVzaW5nOg0KDQogICBTZWN0aW9uIDYuNA0KDQogICBJbiB0aGUg Y29uc2lkZXJlZCBxdWV1aW5nIG1vZGVsLCB3ZSBjb25zaWRlcmVkIHRoZSBmb3VyIHRyYWZmaWMN CiAgIGNsYXNzZXMgKERlZmluaXRpb24gMy4yNjggb2YgW0lFRUU4MDIxUV0pOiBjb250cm9sLWRh dGEgdHJhZmZpYw0KICAgKENEVCksIGNsYXNzIEEsIGNsYXNzIEIsIGFuZCBiZXN0IGVmZm9ydCAo QkUpIGluIGRlY3JlYXNpbmcgb3JkZXIgb2YNCiAgIHByaW9yaXR5LiAgRmxvd3Mgb2YgY2xhc3Nl cyBBIGFuZCBCIGFyZSB0b2dldGhlciByZWZlcnJlZCBhcyBBVkINCiAgIGZsb3dzLiAgVGhpcyBt b2RlbCBpcyBhIHN1YnNldCBvZiBUaW1lLVNlbnNpdGl2ZSBOZXR3b3JraW5nIGFzDQogICBkZXNj cmliZWQgbmV4dC4NCg0KICAgLi4uLg0KDQogICBUaGVuLA0KICAgdGhlIGlucHV0IGZsb3dzIGFy ZSBpZGVudGlmaWVkIHVzaW5nIHRoZSBpbmZvcm1hdGlvbiBpbiAoU2VjdGlvbiA1LjENCiAgIG9m IFtSRkM4OTM5XSkuICBUaGVuIHRoZXkgYXJlIGFnZ3JlZ2F0ZWQgaW50byBlaWdodCBtYWNybyBm bG93cyBiYXNlZA0KICAgb24gdGhlaXIgdHJhZmZpYyBjbGFzc2VzLiAgV2UgcmVmZXIgdG8gZWFj aCBtYWNybyBmbG93IGFzIGEgY2xhc3MuDQoNClNvIGluIHRoZSBmaXJzdCBwYXJhZ3JhcGgsIHlv dSBzYXkgInRyYWZmaWMgY2xhc3NlcyIgaXMgZGVmaW5lZCBieSA4MDIuMVEuICBpbiB0aGUgc2Vj b25kLCB5b3Ugc2F5ICJ0aGVpciB0cmFmZmljIGNsYXNzZXMiIGlzIGRlZmluZWQgYmFzZWQgb24g W1JGQzg5MzldLiAgSSBiZWxpZXZlIHRoZXNlIGFyZSBpbmNvbnNpc3RlbnQgZGVmaW5pdGlvbnMu ICBJIHRoaW5rIHlvdSBhcmUgdHJ5aW5nIHRvIHNheSB0aGF0DQoNCiAgICBUaGVuIHRoZXkgYXJl IGFnZ3JlZ2F0ZWQgaW50byBlaWdodCBtYWNybyBmbG93cyBiYXNlZCBvbiB0aGVpciAqRGV0TmV0 IGZsb3cgYWdncmVnYXRlLioNCg0KVGhpcyBpcyBhc3N1bWluZyB0aGF0IHRoZSBub2RlIGlzbid0 IGRvaW5nIHNvbWUgc29ydCBvZiBpbmRlcGVuZGVudCBtYXBwaW5nIG9mIG1pY3JvLWZsb3dzIHRv IG1hY3JvLWZsb3dzLg0KDQpQbGVhc2UgbGV0IG1lIGtub3cgaWYgSSdtIG1pc3Npbmcgc29tZXRo aW5nLg0KDQpMb3UNCg0KT24gMy8yMi8yMDIxIDExOjMzIEFNLCBNb2hhbW1hZHBvdXIgRWhzYW4g d3JvdGU6DQoNCkRlYXIgV0csDQoNCldlIHN1Ym1pdHRlZCBhIG5ldyB2ZXJzaW9uIG9mIHRoZSBC b3VuZGVkIExhdGVuY3kgZHJhZnQgdGhhdCBhZGRyZXNzZXMgdGhlIGNvbW1lbnRzIHJlY2VpdmVk IGluIHRoZSBsaXN0IChodHRwczovL2RhdGF0cmFja2VyLmlldGYub3JnL2RvYy9odG1sL2RyYWZ0 LWlldGYtZGV0bmV0LWJvdW5kZWQtbGF0ZW5jeS0wNDxodHRwczovL3Rvb2xzLmlldGYub3JnL2h0 bWwvZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTAzPik7IGl0IGlzIG5vdyByZWFk eSBmb3Igc3VibWlzc2lvbiB0byB0aGUgSUVTRyBmb3IgcHVibGljYXRpb24uDQoNClRoYW5rcyBM b3UgZm9yIGhpcyBjb21tZW50cy4gSW4gdGhlIG5ldyB2ZXJzaW9uLCB3ZSBhZGRyZXNzIHRoZSBj b21tZW50cyBhcyBmb2xsb3dzOg0KDQotIEFic3RyYWN0DQpJIHRoaW5rIHRoZSBhYnN0cmFjdCBz aG91bGQgdG8gYmUgdXBkYXRlZCB0byBtYXRjaCB0aGUgY3VycmVudCBjb250ZW50IG9mIHRoZSBk b2N1bWVudC4gSSBzdWdnZXN0IGp1c3QgY29weWluZyB0aGUgNHRoIGFuZCA1dGggcGFyYWdyYXBo cyBmcm9tIHRoZSBJbnRyb2R1Y3Rpb24gc2VjdGlvbi4NCg0KDQpUaGFua3MgZm9yIHRoZSBzdWdn ZXN0aW9uLiBUaGUgbmV3IHZlcnNpb24gaXMgdXBkYXRlZCBhY2NvcmRpbmdseS4NCg0KLSBnZW5l cmFsIGNvbW1lbnQ6IGNsYXNzZXMgb2YgKERldE5ldCkgU2VydmljZQ0KDQpUaGUgZG9jdW1lbnQg c3RhdGVzIHRoYXQgaXQgdXNlcyB0ZXJtaW5vbG9neSBmcm9tIFJGQzg2NTUsIGl0IGFsc28gdXNl cyB0aGUgdGVybXMgIkNsYXNzZXMgb2YgRGV0TmV0IFNlcnZpY2UiLCAiIERldE5ldCBDbGFzc2Vz IG9mIFNlcnZpY2UiICBhbmQgb3RoZXIgc2ltaWxhciBmb3JtcyBvZiB0aGVzZSB0ZXJtcy4gIFJG Qzg2NTUgdXNlcyB0aGlzIHRlcm0gaW4gb25lIHBsYWNlICIgQ2xhc3Mtb2YtU2VydmljZSBzY2hl bWVzIChlLmcuLCBEaWZmU2VydikiIGFuZCBhbHNvIG1lbnRpb25zICJjbGFzc2VzIG9mIERldE5l dCBmbG93cyIgIGluIHR3byBwbGFjZXMuICBUaGUgZGF0YSBwbGFuZSBkb2N1bWVudHMgW1JGQzg5 NjRdIGFuZCBbUkZDODkzNF0gYWxzbyBkZXNjcmliZSBDbGFzcyBvZiBTZXJ2aWNlIGluIHRoZSBj b250ZXh0IG9mIERpZmZTZXJ2IChhbmQgbm90IERldE5ldCkuDQoNCkluIHJlYWRpbmcgdGhlIGRv Y3VtZW50IGl0IHNlZW1zIHRoYXQgaXQgaXMgbm90IHVzaW5nIENvUyBpbiB0aGUgdHlwaWNhbCB3 YXkgdXNlZCBpbiB0aGUgSUVURiAgb3IgdGhlIG90aGVyIERldE5ldCBSRkNzLCBhbmQgaXMgaW50 cm9kdWNpbmcgYSBuZXcgZGVmaW5pdGlvbiBmb3IgQ29TLiAgIElJIHRoaW5rIGhhdmluZyBhIGRv Y3VtZW50IHNwZWNpZmljIGRlZmluaXRpb24gb2YgQ29TIHByb2JhYmx5IGlzbid0IGhlbHBmdWwu IEl0IHdvdWxkIGJlIGJlc3QgdG8gdXNlIGV4aXN0aW5nIERldE5ldCB0ZXJtaW5vbG9neS4gIEkg c3VzcGVjdCAgImZsb3cgYWdncmVnYXRlIiBpcyB0aGUgY2xvc2VzdCwgYnV0IHBlcmhhcHMgSSdt IG1pc3Npbmcgc29tZXRoaW5nLg0KDQoNClRoYW5rcyBmb3IgeW91ciBjb21tZW50LiBJbiB0aGUg bmV3IHZlcnNpb24sIHdlIHVzZWQgdGhlIHRlcm0g4oCcYWdncmVnYXRl4oCdIGluc3RlYWQgb2Yg Y2xhc3MgaW4gc2VjdGlvbiAzLjEgYW5kIGNsYXNzZXMgb2YgRGV0TmV0IGZsb3dzLiBTZWN0aW9u IDYuNCB1c2VzIHRoZSB0ZXJtIOKAnHRyYWZmaWMgY2xhc3PigJ0gYXMgaW4gSUVFRTgwMi4xUS4N Cg0KLSBhbGlnbm1lbnQgd2l0aCB0aGUgRGF0YSBQbGFuZSBSRkNzDQoNClRoZSBsaXN0cyBpbiBz ZWN0aW9uIDMuMSBhbmQgaW4gc2VjdGlvbiA2LjQgYXJlIG5vdCB3ZWxsIGFsaWduZWQgd2l0aCB0 aGUgRGF0YSBwbGFuZSBkb2N1bWVudHMuICBUaGVzZSBzZWN0aW9ucyBzaG91bGQgYmUgYWxpZ25l ZCB3aXRoIHRoZSBwcm92aXNpb25pbmcgb2YgSVAgZmxvd3MsIGFzIHN1bW1hcml6ZWQgaW4gU2Vj dGlvbiAgNiBvZiBSRkM4OTM5LCBhbmQgTVBMUyBhcyBzdW1tYXJpemVkIGluIFNlY3Rpb24gNSBv ZiBSRkM4OTY0Lg0KDQpUaGFua3MgZm9yIHlvdXIgY29tbWVudC4gU2VjdGlvbiAzLjEgaXMgYWJv dXQgZmxvdyBhZG1pc3Npb24gcGFyYWRpZ20gYW5kIG5vdCAiZmxvdyBjcmVhdGlvbuKAnS4gVGhl IHRpdGxlIGlzIGNoYW5nZWQgdG8g4oCcZmxvdyBhZG1pc3Npb27igJ0gdG8gYXZvaWQgcG9zc2li bGUgY29uZnVzaW9uLiBJbiBmYWN0LCB0aGlzIHNlY3Rpb24gaXMgYWxpZ25lZCB3aXRoIHRoZSBE YXRhUGxhbmUgUkZDcyBhbmQgaW4gcGFyYWxsZWwgd2l0aCBwcm92aXNpb25pbmcgb2YgSVAgZmxv d3MuIFdlIG1vZGlmaWVkIHNlY3Rpb24gNi40IHRvIGZvbGxvdyB0aGUgRGF0YSBwbGFuZSBkb2N1 bWVudHMuDQoNCg0KLSBtaW5vciBjb21tZW50IHNlcnZpY2UgdnMgZnJhbWUgcHJlZW1wdGlvbg0K DQpTZWN0aW9uIDMuMSB1c2VzIHRoZSB0ZXJtICJ1bi1wcm92aXNpb25pbmciLCBJIHJlYWQgdGhl IHRleHQgYXMgbWVhbmluZyAic2VydmljZSBwcmVlbXB0aW9uIi4NCg0KSW4gdGhlIHJlc3Qgb2Yg dGhlIGRvY3VtZW50IHByZWVtcHRpb24gcmVmZXJzIHRvICJmcmFtZSBwcmVlbXB0aW9uIiBhbmQg dGhpcyBzaG91bGQgYmUgY2xhcmlmaWVkLg0KDQpUaGFua3MgZm9yIHRoZSBzdWdnZXN0aW9uLiBJ biB0aGUgbmV3IHZlcnNpb24sIHdlIHVzZSB0aGUgdGVybSDigJxzZXJ2aWNlIHByZWVtcHRpb27i gJ0gYW5kIHVwZGF0ZSB0aGUgdGVybSDigJxwcmVlbXB0aW9u4oCdIHRvIOKAnGZyYW1lIHByZWVt cHRpb27igJ0uDQoNCg0KLSBpZCBuaXRzLCBwbGVhc2UgZW5zdXJlIHRoZSBkb2N1bWVudCBwYXNz ZXMgaWRuaXRzIHdpdGhvdXQgZXJyb3JzLCBzZWUNCg0KaHR0cHM6Ly93d3c2LmlldGYub3JnL3Rv b2xzL2lkbml0cz91cmw9aHR0cHM6Ly93d3cuaWV0Zi5vcmcvYXJjaGl2ZS9pZC9kcmFmdC1pZXRm LWRldG5ldC1ib3VuZGVkLWxhdGVuY3ktMDIudHh0DQoNClRoYW5rcyBmb3IgeW91ciBjb21tZW50 IGFib3V0IGlkbml0cy4gV2UgYWRkZWQgdGhlIG1pc3Npbmcgc2VjdGlvbnMsIGkuZS4sIOKAnHNl Y3VyaXR5IGNvbnNpZGVyYXRpb25z4oCdIGFuZCDigJxJQU5BIGNvbnNpZGVyYXRpb25z4oCdIHRv IHRoZSBkcmFmdC4gQWxzbyByZXNvbHZlZCBvdGhlciBjb21tZW50cyBieSBpZG5pdHMuDQoNCkJl c3QsDQpFaHNhbg0KDQoNCg0KDQotLQ0KRWhzYW4gTW9oYW1tYWRwb3VyDQpQaEQgQ2FuZGlkYXRl IGF0IFN3aXNzIEZlZGVyYWwgSW5zdGl0dXRlIG9mIFRlY2hub2xvZ3kgKEVQRkwpDQpFUEZMIEVE SUMgUGhEIHN0dWRlbnQgcmVwcmVzZW50YXRpdmUNCklDIElJTkZDT00sIExDQTIsIElORiAwMTEs IFN0YXRpb24gMTQsIDEwMTUgTGF1c2FubmUsIFN3aXR6ZXJsYW5kDQpodHRwczovL3Blb3BsZS5l cGZsLmNoL2Voc2FuLm1vaGFtbWFkcG91cjxodHRwczovL3Blb3BsZS5lcGZsLmNoL2Voc2FuLm1v aGFtbWFkcG91cj9sYW5nPWVuPg0KDQoNCk9uIDIxIEZlYiAyMDIxLCBhdCAyMzoxNSwgTG91IEJl cmdlciA8bGJlcmdlckBsYWJuLm5ldDxtYWlsdG86bGJlcmdlckBsYWJuLm5ldD4+IHdyb3RlOg0K DQpBbGwsDQoNClRoZSBXRyBsYXN0IGNhbGwgaXMgY29tcGxldGUuDQoNCkF1dGhvcnMsDQoNClBs ZWFzZSBicmluZyBhbnkgcmVzb2x1dGlvbnMgdG8gY29tbWVudHMgcmVjZWl2ZWQgdG8gdGhlIGxp c3QuIE9uY2UgYWxsIGNvbW1lbnRzIGFyZSBhZGRyZXNzZWQgcGxlYXNlIHVwZGF0ZSB0aGUgZHJh ZnQgYW5kIGxldCB0aGUgV0cga25vdyB0aGF0IGl0IGlzIHJlYWR5IGZvciBzdWJtaXNzaW9uIHRv IHRoZSBJRVNHIGZvciBwdWJsaWNhdGlvbnMuDQoNClRoYW5rIHlvdSENCg0KTG91DQoNCk9uIDIv NS8yMDIxIDg6MzAgQU0sIExvdSBCZXJnZXIgd3JvdGU6DQpBbGwsDQoNClRoaXMgc3RhcnRzIHdv cmtpbmcgZ3JvdXAgbGFzdCBjYWxsIG9uIGRyYWZ0LWlldGYtZGV0bmV0LWJvdW5kZWQtbGF0ZW5j eS0wMg0KaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtaWV0Zi1kZXRuZXQt Ym91bmRlZC1sYXRlbmN5Lw0KDQpUaGUgd29ya2luZyBncm91cCBsYXN0IGNhbGwgZW5kcyBvbiBG ZWJydWFyeSAxOXRoLg0KUGxlYXNlIHNlbmQgeW91ciBjb21tZW50cyB0byB0aGUgd29ya2luZyBn cm91cCBtYWlsaW5nIGxpc3QuDQoNClBsZWFzZSBub3RlLCB0aGVyZSB3YXMgYW4gSVBSIGRpc2Ns b3N1cmUgYWdhaW5zdCB0aGlzIGRvY3VtZW50DQpzdWJtaXR0ZWQgb24gSmFudWFyeSAxNCwgMjAy MSwgc2VlIGh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvaXByLzQ1ODEvDQpXaGlsZSB0aGlz IGRpc2Nsb3N1cmUgY2FtZSB2ZXJ5IGxhdGUgaW4gdGhlIGRvY3VtZW50IGxpZmUgY3ljbGUsDQpp dCBhcHBlYXJzIHRoZSBmaWxpbmcgd2FzIGFsc28gcmVsYXRpdmVseSByZWNlbnQgKDIwMTktMTAt MTYpIGFuZA0KYWZ0ZXIgdGhlIHByZSBhZG9wdGlvbiBJUCBjYWxsOg0KaHR0cHM6Ly9tYWlsYXJj aGl2ZS5pZXRmLm9yZy9hcmNoL21zZy9kZXRuZXQvMW13WWRhZER6VEx1ZGN1SDVUNU83Ul9LbERz DQoNClBvc2l0aXZlIGNvbW1lbnRzLCBlLmcuLCAiSSd2ZSByZXZpZXdlZCB0aGlzIGRvY3VtZW50 DQphbmQgYmVsaWV2ZSBpdCBpcyByZWFkeSBmb3IgcHVibGljYXRpb24iLCBhcmUgd2VsY29tZSEN ClRoaXMgaXMgdXNlZnVsIGFuZCBpbXBvcnRhbnQsIGV2ZW4gZnJvbSBhdXRob3JzLg0KDQpUaGFu ayB5b3UsDQpMb3UgKERldE5ldCBDby1DaGFpciAmIGRvYyBTaGVwaGVyZCkNCg0KDQoNCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpkZXRuZXQgbWFpbGlu ZyBsaXN0DQpkZXRuZXRAaWV0Zi5vcmc8bWFpbHRvOmRldG5ldEBpZXRmLm9yZz4NCmh0dHBzOi8v d3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vZGV0bmV0DQoNCg0K --_000_6D2EFDE8452C40C7B4AF5300B995FE2Bepflch_ Content-Type: text/html; charset="utf-8" Content-ID: <9B30BE4060FD8D42BB4C5B1CC3EEAC67@intranet.epfl.ch> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0 ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCkhlbGxvIExvdSwNCjxkaXYgY2xhc3M9IiI+PGJy IGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRoYW5rcyBmb3IgdGhlIGNvbW1lbnQs IHlvdeKAmXJlIHJpZ2h0ISBJIHdpbGwgdXNlIHRoZSBmb2xsb3dpbmcgc2VudGVuY2U6PC9kaXY+ DQo8ZGl2IGNsYXNzPSIiPg0KPGRpdiBjbGFzcz0iIj4NCjxkaXYgY2xhc3M9IiI+DQo8cCBjbGFz cz0iIj4mcXVvdDtUaGVuIHRoZXkgYXJlIGFnZ3JlZ2F0ZWQgaW50byBlaWdodCBtYWNybyBmbG93 cyBiYXNlZCBvbiB0aGVpciBEZXROZXQgZmxvdyBhZ2dyZWdhdGUuJnF1b3Q7PC9wPg0KPC9kaXY+ DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9ImNhcmV0LWNv bG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGxldHRlci1zcGFjaW5nOiBu b3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9y bTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQt dGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyB3b3JkLXdyYXA6 IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0ZXIt d2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHlsZT0iY2FyZXQtY29s b3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNwYWNpbmc6IG5v cm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3Jt OiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10 ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdvcmQtd3JhcDog YnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBhZnRlci13 aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xv cjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9y bWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06 IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRl eHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBi cmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdo aXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9ImNhcmV0LWNvbG9y OiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGxldHRlci1zcGFjaW5nOiBub3Jt YWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTog bm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4 dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyB3b3JkLXdyYXA6IGJy ZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0ZXItd2hp dGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHlsZT0iY2FyZXQtY29sb3I6 IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNwYWNpbmc6IG5vcm1h bDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBu b25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0 LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdvcmQtd3JhcDogYnJl YWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBhZnRlci13aGl0 ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xvcjog cmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFs OyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5v bmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQt c3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVh ay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRl LXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9ImNhcmV0LWNvbG9yOiBy Z2IoMCwgMCwgMCk7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7 IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9u ZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1z dHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyB3b3JkLXdyYXA6IGJyZWFr LXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0ZXItd2hpdGUt c3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHlsZT0iY2FyZXQtY29sb3I6IHJn YigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsg dGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25l OyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0 cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdvcmQtd3JhcDogYnJlYWst d29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBhZnRlci13aGl0ZS1z cGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xvcjogcmdi KDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0 ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7 IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ry b2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVhay13 b3JkOyAtd2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNw YWNlOyIgY2xhc3M9IiI+DQo8ZGl2IHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAsIDAsIDApOyBj b2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogSGVsdmV0aWNhOyBmb250LXNpemU6IDEy cHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWNhcHM6IG5vcm1hbDsgZm9udC13 ZWlnaHQ6IG5vcm1hbDsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7 IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9y bWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0 ZXh0LWRlY29yYXRpb246IG5vbmU7Ij4NCjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBzdHls ZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1m YW1pbHk6IEhlbHZldGljYTsgZm9udC1zaXplOiAxMnB4OyBmb250LXN0eWxlOiBub3JtYWw7IGZv bnQtdmFyaWFudC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGxldHRlci1zcGFj aW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRy YW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13 ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyI+DQpC ZXN0LDwvZGl2Pg0KPGRpdiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6 IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEhlbHZldGljYTsgZm9udC1zaXplOiAxMnB4OyBm b250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0 OiBub3JtYWw7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0 LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsg d29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1k ZWNvcmF0aW9uOiBub25lOyI+DQpFaHNhbjwvZGl2Pg0KPGRpdiBzdHlsZT0iY2FyZXQtY29sb3I6 IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEhlbHZldGlj YTsgZm9udC1zaXplOiAxMnB4OyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBz OiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRl eHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsg d2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJv a2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyI+DQo8YnIgY2xhc3M9IiI+DQo8 L2Rpdj4NCjxkaXYgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2Io MCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBIZWx2ZXRpY2E7IGZvbnQtc2l6ZTogMTJweDsgZm9udC1z dHlsZTogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXdlaWdodDogbm9y bWFsOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRl bnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQt c3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3Jh dGlvbjogbm9uZTsiPg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IHN0eWxlPSJjYXJldC1j b2xvcjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogSGVs dmV0aWNhOyBmb250LXNpemU6IDEycHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50 LWNhcHM6IG5vcm1hbDsgZm9udC13ZWlnaHQ6IG5vcm1hbDsgbGV0dGVyLXNwYWNpbmc6IG5vcm1h bDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBu b25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0 LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7Ij4NClAuUy4gVGhlIGlu dGVudGlvbiBvZiAmcXVvdDtUaGVuLCB0aGUgaW5wdXQgZmxvd3MgYXJlIGlkZW50aWZpZWQgdXNp bmcgdGhlIGluZm9ybWF0aW9uIGluIChTZWN0aW9uIDUuMSBvZiBbUkZDODkzOV0p4oCdLCB3YXMg dG8gc2F5IHRoYXQgd2UgdXNlIFNlY3Rpb24gNS4xIG9mIFJGQzg5MzkgdG8gaWRlbnRpZnkgdGhl IGluZGl2aWR1YWwgRGV0TmV0IGZsb3dzLjwvZGl2Pg0KPGRpdiBzdHlsZT0iY2FyZXQtY29sb3I6 IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEhlbHZldGlj YTsgZm9udC1zaXplOiAxMnB4OyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBz OiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRl eHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsg d2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJv a2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyI+DQo8YnIgY2xhc3M9IiI+DQo8 L2Rpdj4NCjxkaXYgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGNvbG9yOiByZ2Io MCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBIZWx2ZXRpY2E7IGZvbnQtc2l6ZTogMTJweDsgZm9udC1z dHlsZTogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXdlaWdodDogbm9y bWFsOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRl bnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQt c3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3Jh dGlvbjogbm9uZTsiPg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IHN0eWxlPSJjYXJldC1j b2xvcjogcmdiKDAsIDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogSGVs dmV0aWNhOyBmb250LXNpemU6IDEycHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50 LWNhcHM6IG5vcm1hbDsgZm9udC13ZWlnaHQ6IG5vcm1hbDsgbGV0dGVyLXNwYWNpbmc6IG5vcm1h bDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBu b25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0 LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7Ij4NCjxiciBjbGFzcz0i Ij4NCjwvZGl2Pg0KPGRpdiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6 IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEhlbHZldGljYTsgZm9udC1zaXplOiAxMnB4OyBm b250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0 OiBub3JtYWw7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0 LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsg d29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1k ZWNvcmF0aW9uOiBub25lOyI+DQotLTxiciBjbGFzcz0iIj4NCkVoc2FuIE1vaGFtbWFkcG91cjwv ZGl2Pg0KPGRpdiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgY29sb3I6IHJnYigw LCAwLCAwKTsgZm9udC1mYW1pbHk6IEhlbHZldGljYTsgZm9udC1zaXplOiAxMnB4OyBmb250LXN0 eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3Jt YWw7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVu dDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1z cGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0 aW9uOiBub25lOyI+DQpQaEQgQ2FuZGlkYXRlIGF0IFN3aXNzIEZlZGVyYWwgSW5zdGl0dXRlIG9m IFRlY2hub2xvZ3kgKEVQRkwpPC9kaXY+DQo8ZGl2IHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAs IDAsIDApOyBjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogSGVsdmV0aWNhOyBmb250 LXNpemU6IDEycHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWNhcHM6IG5vcm1h bDsgZm9udC13ZWlnaHQ6IG5vcm1hbDsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4dC1hbGln bjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1z cGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0 aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7Ij4NCklDIElJTkZDT00sIExDQTIsIElORiAw MTEsIFN0YXRpb24gMTQsJm5ic3A7MTAxNSBMYXVzYW5uZSwgU3dpdHplcmxhbmQ8YnIgY2xhc3M9 IiI+DQo8YSBocmVmPSJodHRwczovL3Blb3BsZS5lcGZsLmNoL2Voc2FuLm1vaGFtbWFkcG91cj9s YW5nPWVuIiBjbGFzcz0iIj5odHRwczovL3Blb3BsZS5lcGZsLmNoL2Voc2FuLm1vaGFtbWFkcG91 cjwvYT4mbmJzcDs8YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4N CjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0K PGRpdj48YnIgY2xhc3M9IiI+DQo8YmxvY2txdW90ZSB0eXBlPSJjaXRlIiBjbGFzcz0iIj4NCjxk aXYgY2xhc3M9IiI+T24gMjIgTWFyIDIwMjEsIGF0IDE2OjU2LCBMb3UgQmVyZ2VyICZsdDs8YSBo cmVmPSJtYWlsdG86bGJlcmdlckBsYWJuLm5ldCIgY2xhc3M9IiI+bGJlcmdlckBsYWJuLm5ldDwv YT4mZ3Q7IHdyb3RlOjwvZGl2Pg0KPGJyIGNsYXNzPSJBcHBsZS1pbnRlcmNoYW5nZS1uZXdsaW5l Ij4NCjxkaXYgY2xhc3M9IiI+DQo8ZGl2IGNsYXNzPSIiPg0KPHAgY2xhc3M9IiI+RWhzYW4gPGJy IGNsYXNzPSIiPg0KPC9wPg0KPHAgY2xhc3M9IiI+VGhhbmsgeW91IGZvciB0aGUgdXBkYXRlLiZu YnNwOyBJIGRpZCBhIHF1aWNrIHNraW0gYW5kIGl0IGxvb2tzIG11Y2ggaW1wcm92ZWQuJm5ic3A7 IEkgZm91bmQgb25lIGJpdCBvZiBuZXcgbGFuZ3VhZ2UgY29uZnVzaW5nOjxiciBjbGFzcz0iIj4N CjwvcD4NCjxwIGNsYXNzPSIiPiZuYnNwOyZuYnNwOyBTZWN0aW9uIDYuNDwvcD4NCjxwIGNsYXNz PSIiPiZuYnNwOyZuYnNwOyBJbiB0aGUgY29uc2lkZXJlZCBxdWV1aW5nIG1vZGVsLCB3ZSBjb25z aWRlcmVkIHRoZSBmb3VyIHRyYWZmaWM8YnIgY2xhc3M9IiI+DQombmJzcDsmbmJzcDsgY2xhc3Nl cyAoRGVmaW5pdGlvbiAzLjI2OCBvZiBbSUVFRTgwMjFRXSk6IGNvbnRyb2wtZGF0YSB0cmFmZmlj PGJyIGNsYXNzPSIiPg0KJm5ic3A7Jm5ic3A7IChDRFQpLCBjbGFzcyBBLCBjbGFzcyBCLCBhbmQg YmVzdCBlZmZvcnQgKEJFKSBpbiBkZWNyZWFzaW5nIG9yZGVyIG9mPGJyIGNsYXNzPSIiPg0KJm5i c3A7Jm5ic3A7IHByaW9yaXR5LiZuYnNwOyBGbG93cyBvZiBjbGFzc2VzIEEgYW5kIEIgYXJlIHRv Z2V0aGVyIHJlZmVycmVkIGFzIEFWQjxiciBjbGFzcz0iIj4NCiZuYnNwOyZuYnNwOyBmbG93cy4m bmJzcDsgVGhpcyBtb2RlbCBpcyBhIHN1YnNldCBvZiBUaW1lLVNlbnNpdGl2ZSBOZXR3b3JraW5n IGFzPGJyIGNsYXNzPSIiPg0KJm5ic3A7Jm5ic3A7IGRlc2NyaWJlZCBuZXh0LjwvcD4NCjxwIGNs YXNzPSIiPiZuYnNwOyZuYnNwOyAuLi4uPGJyIGNsYXNzPSIiPg0KPC9wPg0KPHAgY2xhc3M9IiI+ Jm5ic3A7Jm5ic3A7IFRoZW4sPGJyIGNsYXNzPSIiPg0KJm5ic3A7Jm5ic3A7IHRoZSBpbnB1dCBm bG93cyBhcmUgaWRlbnRpZmllZCB1c2luZyB0aGUgaW5mb3JtYXRpb24gaW4gKFNlY3Rpb24gNS4x PGJyIGNsYXNzPSIiPg0KJm5ic3A7Jm5ic3A7IG9mIFtSRkM4OTM5XSkuJm5ic3A7IFRoZW4gdGhl eSBhcmUgYWdncmVnYXRlZCBpbnRvIGVpZ2h0IG1hY3JvIGZsb3dzIGJhc2VkPGJyIGNsYXNzPSIi Pg0KJm5ic3A7Jm5ic3A7IG9uIHRoZWlyIHRyYWZmaWMgY2xhc3Nlcy4mbmJzcDsgV2UgcmVmZXIg dG8gZWFjaCBtYWNybyBmbG93IGFzIGEgY2xhc3MuPC9wPg0KPHAgY2xhc3M9IiI+U28gaW4gdGhl IGZpcnN0IHBhcmFncmFwaCwgeW91IHNheSAmcXVvdDt0cmFmZmljIGNsYXNzZXMmcXVvdDsgaXMg ZGVmaW5lZCBieSA4MDIuMVEuJm5ic3A7IGluIHRoZSBzZWNvbmQsIHlvdSBzYXkgJnF1b3Q7dGhl aXIgdHJhZmZpYyBjbGFzc2VzJnF1b3Q7IGlzIGRlZmluZWQgYmFzZWQgb24gW1JGQzg5MzldLiZu YnNwOyBJIGJlbGlldmUgdGhlc2UgYXJlIGluY29uc2lzdGVudCBkZWZpbml0aW9ucy4mbmJzcDsg SSB0aGluayB5b3UgYXJlIHRyeWluZyB0byBzYXkgdGhhdDwvcD4NCjxwIGNsYXNzPSIiPiZuYnNw OyZuYnNwOyZuYnNwOyBUaGVuIHRoZXkgYXJlIGFnZ3JlZ2F0ZWQgaW50byBlaWdodCBtYWNybyBm bG93cyBiYXNlZCBvbiB0aGVpciAqRGV0TmV0IGZsb3cgYWdncmVnYXRlLio8L3A+DQo8cCBjbGFz cz0iIj5UaGlzIGlzIGFzc3VtaW5nIHRoYXQgdGhlIG5vZGUgaXNuJ3QgZG9pbmcgc29tZSBzb3J0 IG9mIGluZGVwZW5kZW50IG1hcHBpbmcgb2YgbWljcm8tZmxvd3MgdG8gbWFjcm8tZmxvd3MuPGJy IGNsYXNzPSIiPg0KPC9wPg0KPHAgY2xhc3M9IiI+UGxlYXNlIGxldCBtZSBrbm93IGlmIEknbSBt aXNzaW5nIHNvbWV0aGluZy48L3A+DQo8cCBjbGFzcz0iIj5Mb3U8YnIgY2xhc3M9IiI+DQo8L3A+ DQo8cCBjbGFzcz0iIj5PbiAzLzIyLzIwMjEgMTE6MzMgQU0sIE1vaGFtbWFkcG91ciBFaHNhbiB3 cm90ZTo8YnIgY2xhc3M9IiI+DQo8L3A+DQo8YmxvY2txdW90ZSB0eXBlPSJjaXRlIiBjaXRlPSJt aWQ6ODYwODk0MUYtQUQ0NC00MDk4LTgzREYtNDE2RkJEMkRGRUEzQGVwZmwuY2giIGNsYXNzPSIi Pg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJ3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQt bmJzcC1tb2RlOg0KICAgICAgICBzcGFjZTsgbGluZS1icmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7 IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHlsZT0id29yZC13cmFwOiBicmVhay13b3Jk OyAtd2Via2l0LW5ic3AtbW9kZToNCiAgICAgICAgICBzcGFjZTsgbGluZS1icmVhazogYWZ0ZXIt d2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgY2xhc3M9IiI+RGVhciBXRyw8L2Rpdj4NCjxk aXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPldlIHN1Ym1p dHRlZCBhIG5ldyB2ZXJzaW9uIG9mIHRoZSBCb3VuZGVkIExhdGVuY3kgZHJhZnQgdGhhdCBhZGRy ZXNzZXMgdGhlIGNvbW1lbnRzIHJlY2VpdmVkIGluIHRoZSBsaXN0ICg8YSBocmVmPSJodHRwczov L3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTAz IiBjbGFzcz0iIiBtb3otZG8tbm90LXNlbmQ9InRydWUiPmh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0 Zi5vcmcvZG9jL2h0bWwvZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTA0PC9hPik7 DQogaXQgaXMgbm93IHJlYWR5IGZvciBzdWJtaXNzaW9uIHRvIHRoZSBJRVNHIGZvciBwdWJsaWNh dGlvbi48L2Rpdj4NCjxkaXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNs YXNzPSIiPg0KPGRpdiBjbGFzcz0iIj5UaGFua3MgTG91IGZvciBoaXMgY29tbWVudHMuIEluIHRo ZSBuZXcgdmVyc2lvbiwgd2UgYWRkcmVzcyB0aGUgY29tbWVudHMgYXMgZm9sbG93czo8L2Rpdj4N CjxkaXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPg0KPGJs b2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9IiI+LSBBYnN0cmFjdDxiciBjbGFzcz0iIj4NCkkg dGhpbmsgdGhlIGFic3RyYWN0IHNob3VsZCB0byBiZSB1cGRhdGVkIHRvIG1hdGNoIHRoZSBjdXJy ZW50IGNvbnRlbnQgb2YgdGhlIGRvY3VtZW50LiBJIHN1Z2dlc3QganVzdCBjb3B5aW5nIHRoZSA0 dGggYW5kIDV0aCBwYXJhZ3JhcGhzIGZyb20gdGhlIEludHJvZHVjdGlvbiBzZWN0aW9uLjxiciBj bGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjwvYmxvY2txdW90ZT4NCjxkaXYgY2xhc3M9IiI+PGJy IGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRoYW5rcyBmb3IgdGhlIHN1Z2dlc3Rp b24uIFRoZSBuZXcgdmVyc2lvbiBpcyB1cGRhdGVkIGFjY29yZGluZ2x5LjwvZGl2Pg0KPGJyIGNs YXNzPSIiPg0KPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9IiI+LSBnZW5lcmFsIGNvbW1l bnQ6IGNsYXNzZXMgb2YgKERldE5ldCkgU2VydmljZTxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0i Ij4NClRoZSBkb2N1bWVudCBzdGF0ZXMgdGhhdCBpdCB1c2VzIHRlcm1pbm9sb2d5IGZyb20gUkZD ODY1NSwgaXQgYWxzbyB1c2VzIHRoZSB0ZXJtcyAmcXVvdDtDbGFzc2VzIG9mIERldE5ldCBTZXJ2 aWNlJnF1b3Q7LCAmcXVvdDsgRGV0TmV0IENsYXNzZXMgb2YgU2VydmljZSZxdW90OyZuYnNwOyBh bmQgb3RoZXIgc2ltaWxhciBmb3JtcyBvZiB0aGVzZSB0ZXJtcy4mbmJzcDsgUkZDODY1NSB1c2Vz IHRoaXMgdGVybSBpbiBvbmUgcGxhY2UgJnF1b3Q7IENsYXNzLW9mLVNlcnZpY2Ugc2NoZW1lcyAo ZS5nLiwgRGlmZlNlcnYpJnF1b3Q7DQogYW5kIGFsc28gbWVudGlvbnMgJnF1b3Q7Y2xhc3NlcyBv ZiBEZXROZXQgZmxvd3MmcXVvdDsmbmJzcDsgaW4gdHdvIHBsYWNlcy4mbmJzcDsgVGhlIGRhdGEg cGxhbmUgZG9jdW1lbnRzIFtSRkM4OTY0XSBhbmQgW1JGQzg5MzRdIGFsc28gZGVzY3JpYmUgQ2xh c3Mgb2YgU2VydmljZSBpbiB0aGUgY29udGV4dCBvZiBEaWZmU2VydiAoYW5kIG5vdCBEZXROZXQp LjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCkluIHJlYWRpbmcgdGhlIGRvY3VtZW50IGl0 IHNlZW1zIHRoYXQgaXQgaXMgbm90IHVzaW5nIENvUyBpbiB0aGUgdHlwaWNhbCB3YXkgdXNlZCBp biB0aGUgSUVURiZuYnNwOyBvciB0aGUgb3RoZXIgRGV0TmV0IFJGQ3MsIGFuZCBpcyBpbnRyb2R1 Y2luZyBhIG5ldyBkZWZpbml0aW9uIGZvciBDb1MuICZuYnNwOyBJSSB0aGluayBoYXZpbmcgYSBk b2N1bWVudCBzcGVjaWZpYyBkZWZpbml0aW9uIG9mIENvUyBwcm9iYWJseSBpc24ndCBoZWxwZnVs LiBJdCB3b3VsZCBiZQ0KIGJlc3QgdG8gdXNlIGV4aXN0aW5nIERldE5ldCB0ZXJtaW5vbG9neS4m bmJzcDsgSSBzdXNwZWN0Jm5ic3A7ICZxdW90O2Zsb3cgYWdncmVnYXRlJnF1b3Q7IGlzIHRoZSBj bG9zZXN0LCBidXQgcGVyaGFwcyBJJ20gbWlzc2luZyBzb21ldGhpbmcuPGJyIGNsYXNzPSIiPg0K PGJyIGNsYXNzPSIiPg0KPC9ibG9ja3F1b3RlPg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+ DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhhbmtzIGZvciB5b3VyIGNvbW1lbnQuIEluIHRoZSBu ZXcgdmVyc2lvbiwgd2UgdXNlZCB0aGUgdGVybSDigJxhZ2dyZWdhdGXigJ0gaW5zdGVhZCBvZiBj bGFzcyBpbiBzZWN0aW9uIDMuMSBhbmQgY2xhc3NlcyBvZiBEZXROZXQgZmxvd3MuIFNlY3Rpb24g Ni40IHVzZXMgdGhlIHRlcm0g4oCcdHJhZmZpYyBjbGFzc+KAnSBhcyBpbiBJRUVFODAyLjFRLiZu YnNwOzwvZGl2Pg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxibG9ja3F1 b3RlIHR5cGU9ImNpdGUiIGNsYXNzPSIiPi0gYWxpZ25tZW50IHdpdGggdGhlIERhdGEgUGxhbmUg UkZDczxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NClRoZSBsaXN0cyBpbiBzZWN0aW9uIDMu MSBhbmQgaW4gc2VjdGlvbiA2LjQgYXJlIG5vdCB3ZWxsIGFsaWduZWQgd2l0aCB0aGUgRGF0YSBw bGFuZSBkb2N1bWVudHMuJm5ic3A7IFRoZXNlIHNlY3Rpb25zIHNob3VsZCBiZSBhbGlnbmVkIHdp dGggdGhlIHByb3Zpc2lvbmluZyBvZiBJUCBmbG93cywgYXMgc3VtbWFyaXplZCBpbiBTZWN0aW9u Jm5ic3A7IDYgb2YgUkZDODkzOSwgYW5kIE1QTFMgYXMgc3VtbWFyaXplZCBpbiBTZWN0aW9uIDUg b2YgUkZDODk2NC48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQo8L2Jsb2NrcXVvdGU+DQo8 ZGl2IGNsYXNzPSIiPjxmb250IGNsYXNzPSIiIGZhY2U9IkhlbHZldGljYSBOZXVlIj5UaGFua3Mg Zm9yIHlvdXIgY29tbWVudC4gU2VjdGlvbiAzLjEgaXMgYWJvdXQgZmxvdyBhZG1pc3Npb24gcGFy YWRpZ20gYW5kIG5vdCAmcXVvdDtmbG93IGNyZWF0aW9u4oCdLiBUaGUgdGl0bGUgaXMgY2hhbmdl ZCB0byZuYnNwO+KAnGZsb3cgYWRtaXNzaW9u4oCdIHRvIGF2b2lkIHBvc3NpYmxlIGNvbmZ1c2lv bi4gSW4gZmFjdCwgdGhpcyBzZWN0aW9uIGlzIGFsaWduZWQgd2l0aCB0aGUNCiBEYXRhUGxhbmUg UkZDcyBhbmQgaW4gcGFyYWxsZWwgd2l0aCBwcm92aXNpb25pbmcgb2YgSVAgZmxvd3MuIFdlIG1v ZGlmaWVkIHNlY3Rpb24gNi40IHRvIGZvbGxvdyZuYnNwO3RoZSBEYXRhIHBsYW5lIGRvY3VtZW50 cy48L2ZvbnQ+PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGJy IGNsYXNzPSIiPg0KPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9IiI+LSBtaW5vciBjb21t ZW50IHNlcnZpY2UgdnMgZnJhbWUgcHJlZW1wdGlvbjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0i Ij4NClNlY3Rpb24gMy4xIHVzZXMgdGhlIHRlcm0gJnF1b3Q7dW4tcHJvdmlzaW9uaW5nJnF1b3Q7 LCBJIHJlYWQgdGhlIHRleHQgYXMgbWVhbmluZyAmcXVvdDtzZXJ2aWNlIHByZWVtcHRpb24mcXVv dDsuPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KSW4gdGhlIHJlc3Qgb2YgdGhlIGRvY3Vt ZW50IHByZWVtcHRpb24gcmVmZXJzIHRvICZxdW90O2ZyYW1lIHByZWVtcHRpb24mcXVvdDsgYW5k IHRoaXMgc2hvdWxkIGJlIGNsYXJpZmllZC48YnIgY2xhc3M9IiI+DQo8L2Jsb2NrcXVvdGU+DQo8 ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaGFua3Mg Zm9yIHRoZSBzdWdnZXN0aW9uLiBJbiB0aGUgbmV3IHZlcnNpb24sIHdlIHVzZSB0aGUgdGVybSDi gJxzZXJ2aWNlIHByZWVtcHRpb27igJ0gYW5kIHVwZGF0ZSB0aGUgdGVybSDigJxwcmVlbXB0aW9u 4oCdIHRvIOKAnGZyYW1lIHByZWVtcHRpb27igJ0uPC9kaXY+DQo8YnIgY2xhc3M9IiI+DQo8Ymxv Y2txdW90ZSB0eXBlPSJjaXRlIiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQotIGlkIG5pdHMsIHBs ZWFzZSBlbnN1cmUgdGhlIGRvY3VtZW50IHBhc3NlcyBpZG5pdHMgd2l0aG91dCBlcnJvcnMsIHNl ZTxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjxhIGhyZWY9Imh0dHBzOi8vd3d3Ni5pZXRm Lm9yZy90b29scy9pZG5pdHM/dXJsPWh0dHBzOi8vd3d3LmlldGYub3JnL2FyY2hpdmUvaWQvZHJh ZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5LTAyLnR4dCIgY2xhc3M9IiIgbW96LWRvLW5v dC1zZW5kPSJ0cnVlIj5odHRwczovL3d3dzYuaWV0Zi5vcmcvdG9vbHMvaWRuaXRzP3VybD1odHRw czovL3d3dy5pZXRmLm9yZy9hcmNoaXZlL2lkL2RyYWZ0LWlldGYtZGV0bmV0LWJvdW5kZWQtbGF0 ZW5jeS0wMi50eHQ8L2E+PC9ibG9ja3F1b3RlPg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2 IGNsYXNzPSIiPlRoYW5rcyBmb3IgeW91ciBjb21tZW50IGFib3V0IGlkbml0cy4gV2UgYWRkZWQg dGhlIG1pc3Npbmcgc2VjdGlvbnMsIGkuZS4sIOKAnHNlY3VyaXR5IGNvbnNpZGVyYXRpb25z4oCd IGFuZCDigJxJQU5BIGNvbnNpZGVyYXRpb25z4oCdIHRvIHRoZSBkcmFmdC4gQWxzbyByZXNvbHZl ZCBvdGhlciBjb21tZW50cyBieSBpZG5pdHMuPC9kaXY+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+ PGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkJlc3QsPC9kaXY+DQo8ZGl2IGNs YXNzPSIiPkVoc2FuPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0K PGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+PGJyIGNs YXNzPSIiPg0KPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRp diBjbGFzcz0iIj4tLTxiciBjbGFzcz0iIj4NCkVoc2FuIE1vaGFtbWFkcG91cjwvZGl2Pg0KPGRp diBjbGFzcz0iIj5QaEQgQ2FuZGlkYXRlIGF0IFN3aXNzIEZlZGVyYWwgSW5zdGl0dXRlIG9mIFRl Y2hub2xvZ3kgKEVQRkwpPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkVQRkwgRURJQyBQaEQgc3R1ZGVu dCByZXByZXNlbnRhdGl2ZSZuYnNwOzxiciBjbGFzcz0iIj4NCklDIElJTkZDT00sIExDQTIsIElO RiAwMTEsIFN0YXRpb24gMTQsJm5ic3A7MTAxNSBMYXVzYW5uZSwgU3dpdHplcmxhbmQ8YnIgY2xh c3M9IiI+DQo8YSBocmVmPSJodHRwczovL3Blb3BsZS5lcGZsLmNoL2Voc2FuLm1vaGFtbWFkcG91 cj9sYW5nPWVuIiBjbGFzcz0iIiBtb3otZG8tbm90LXNlbmQ9InRydWUiPmh0dHBzOi8vcGVvcGxl LmVwZmwuY2gvZWhzYW4ubW9oYW1tYWRwb3VyPC9hPjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj4NCjxk aXYgZGlyPSJhdXRvIiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNw YWNpbmc6IG5vcm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQt dHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsg LXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdv cmQtd3JhcDogYnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFr OiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJj YXJldC1jb2xvcjogcmdiKDAsIDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFs aWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRl LXNwYWNlOiBub3JtYWw7IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdp ZHRoOiAwcHg7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAt d2Via2l0LW5ic3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIg Y2xhc3M9IiI+DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwg MCk7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVu dDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1z cGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0 aW9uOiBub25lOyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFj ZTsgbGluZS1icmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJh dXRvIiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNwYWNpbmc6IG5v cm1hbDsgdGV4dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3Jt OiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10 ZXh0LXN0cm9rZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdvcmQtd3JhcDog YnJlYWstd29yZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBhZnRlci13 aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xv cjogcmdiKDAsIDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFy dDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBu b3JtYWw7IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7 IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5i c3AtbW9kZTogc3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+ DQo8ZGl2IGRpcj0iYXV0byIgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGxldHRl ci1zcGFjaW5nOiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0 ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAw cHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25l OyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1i cmVhazogYWZ0ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgZGlyPSJhdXRvIiBzdHls ZT0iY2FyZXQtY29sb3I6IHJnYigwLCAwLCAwKTsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgdGV4 dC1hbGlnbjogc3RhcnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3 aGl0ZS1zcGFjZTogbm9ybWFsOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9r ZS13aWR0aDogMHB4OyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IHdvcmQtd3JhcDogYnJlYWstd29y ZDsgLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyBsaW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFj ZTsiIGNsYXNzPSIiPg0KPGRpdiBkaXI9ImF1dG8iIHN0eWxlPSJjYXJldC1jb2xvcjogcmdiKDAs IDAsIDApOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1p bmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdv cmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVj b3JhdGlvbjogbm9uZTsgd29yZC13cmFwOiBicmVhay13b3JkOyAtd2Via2l0LW5ic3AtbW9kZTog c3BhY2U7IGxpbmUtYnJlYWs6IGFmdGVyLXdoaXRlLXNwYWNlOyIgY2xhc3M9IiI+DQo8ZGl2IGRp cj0iYXV0byIgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMCwgMCwgMCk7IGxldHRlci1zcGFjaW5n OiBub3JtYWw7IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5z Zm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJr aXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyB3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgbGluZS1icmVhazogYWZ0 ZXItd2hpdGUtc3BhY2U7IiBjbGFzcz0iIj4NCjxkaXYgc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2Io MCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBIZWx2ZXRpY2E7IGZvbnQtc2l6ZTogMTJweDsgZm9udC1z dHlsZTogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXdlaWdodDogbm9y bWFsOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRl bnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdvcmQt c3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IHRleHQtZGVjb3Jh dGlvbjogbm9uZTsiIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8L2Rpdj4NCjwv ZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9k aXY+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+PGJyIGNsYXNzPSIiPg0KPGJsb2NrcXVvdGUgdHlw ZT0iY2l0ZSIgY2xhc3M9IiI+DQo8ZGl2IGNsYXNzPSIiPk9uIDIxIEZlYiAyMDIxLCBhdCAyMzox NSwgTG91IEJlcmdlciAmbHQ7PGEgaHJlZj0ibWFpbHRvOmxiZXJnZXJAbGFibi5uZXQiIGNsYXNz PSIiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+bGJlcmdlckBsYWJuLm5ldDwvYT4mZ3Q7IHdyb3Rl OjwvZGl2Pg0KPGJyIGNsYXNzPSJBcHBsZS1pbnRlcmNoYW5nZS1uZXdsaW5lIj4NCjxkaXYgY2xh c3M9IiI+DQo8ZGl2IGNsYXNzPSIiPkFsbCw8YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpU aGUgV0cgbGFzdCBjYWxsIGlzIGNvbXBsZXRlLjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4N CkF1dGhvcnMsPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KUGxlYXNlIGJyaW5nIGFueSBy ZXNvbHV0aW9ucyB0byBjb21tZW50cyByZWNlaXZlZCB0byB0aGUgbGlzdC4gT25jZSBhbGwgY29t bWVudHMgYXJlIGFkZHJlc3NlZCBwbGVhc2UgdXBkYXRlIHRoZSBkcmFmdCBhbmQgbGV0IHRoZSBX RyBrbm93IHRoYXQgaXQgaXMgcmVhZHkgZm9yIHN1Ym1pc3Npb24gdG8gdGhlIElFU0cgZm9yIHB1 YmxpY2F0aW9ucy48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGFuayB5b3UhPGJyIGNs YXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0KTG91PGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNzPSIiPg0K T24gMi81LzIwMjEgODozMCBBTSwgTG91IEJlcmdlciB3cm90ZTo8YnIgY2xhc3M9IiI+DQo8Ymxv Y2txdW90ZSB0eXBlPSJjaXRlIiBjbGFzcz0iIj5BbGwsPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNz PSIiPg0KVGhpcyBzdGFydHMgd29ya2luZyBncm91cCBsYXN0IGNhbGwgb24gZHJhZnQtaWV0Zi1k ZXRuZXQtYm91bmRlZC1sYXRlbmN5LTAyPGJyIGNsYXNzPSIiPg0KPGEgaHJlZj0iaHR0cHM6Ly9k YXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtaWV0Zi1kZXRuZXQtYm91bmRlZC1sYXRlbmN5 LyIgY2xhc3M9IiIgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIj5odHRwczovL2RhdGF0cmFja2VyLmll dGYub3JnL2RvYy9kcmFmdC1pZXRmLWRldG5ldC1ib3VuZGVkLWxhdGVuY3kvPC9hPjxiciBjbGFz cz0iIj4NCjxiciBjbGFzcz0iIj4NClRoZSB3b3JraW5nIGdyb3VwIGxhc3QgY2FsbCBlbmRzIG9u IEZlYnJ1YXJ5IDE5dGguPGJyIGNsYXNzPSIiPg0KUGxlYXNlIHNlbmQgeW91ciBjb21tZW50cyB0 byB0aGUgd29ya2luZyBncm91cCBtYWlsaW5nIGxpc3QuPGJyIGNsYXNzPSIiPg0KPGJyIGNsYXNz PSIiPg0KUGxlYXNlIG5vdGUsIHRoZXJlIHdhcyBhbiBJUFIgZGlzY2xvc3VyZSBhZ2FpbnN0IHRo aXMgZG9jdW1lbnQ8YnIgY2xhc3M9IiI+DQpzdWJtaXR0ZWQgb24gSmFudWFyeSAxNCwgMjAyMSwg c2VlIDxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHBzOi8vZGF0YXRy YWNrZXIuaWV0Zi5vcmcvaXByLzQ1ODEvIj4NCmh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcv aXByLzQ1ODEvPC9hPjxiciBjbGFzcz0iIj4NCldoaWxlIHRoaXMgZGlzY2xvc3VyZSBjYW1lIHZl cnkgbGF0ZSBpbiB0aGUgZG9jdW1lbnQgbGlmZSBjeWNsZSw8YnIgY2xhc3M9IiI+DQppdCBhcHBl YXJzIHRoZSBmaWxpbmcgd2FzIGFsc28gcmVsYXRpdmVseSByZWNlbnQgKDIwMTktMTAtMTYpIGFu ZDxiciBjbGFzcz0iIj4NCmFmdGVyIHRoZSBwcmUgYWRvcHRpb24gSVAgY2FsbDo8YnIgY2xhc3M9 IiI+DQo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRwczovL21haWxh cmNoaXZlLmlldGYub3JnL2FyY2gvbXNnL2RldG5ldC8xbXdZZGFkRHpUTHVkY3VINVQ1TzdSX0ts RHMiPmh0dHBzOi8vbWFpbGFyY2hpdmUuaWV0Zi5vcmcvYXJjaC9tc2cvZGV0bmV0LzFtd1lkYWRE elRMdWRjdUg1VDVPN1JfS2xEczwvYT48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpQb3Np dGl2ZSBjb21tZW50cywgZS5nLiwgJnF1b3Q7SSd2ZSByZXZpZXdlZCB0aGlzIGRvY3VtZW50PGJy IGNsYXNzPSIiPg0KYW5kIGJlbGlldmUgaXQgaXMgcmVhZHkgZm9yIHB1YmxpY2F0aW9uJnF1b3Q7 LCBhcmUgd2VsY29tZSE8YnIgY2xhc3M9IiI+DQpUaGlzIGlzIHVzZWZ1bCBhbmQgaW1wb3J0YW50 LCBldmVuIGZyb20gYXV0aG9ycy48YnIgY2xhc3M9IiI+DQo8YnIgY2xhc3M9IiI+DQpUaGFuayB5 b3UsPGJyIGNsYXNzPSIiPg0KTG91IChEZXROZXQgQ28tQ2hhaXIgJmFtcDsgZG9jIFNoZXBoZXJk KTxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0iIj4NCjxiciBjbGFzcz0i Ij4NCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyIGNs YXNzPSIiPg0KZGV0bmV0IG1haWxpbmcgbGlzdDxiciBjbGFzcz0iIj4NCjxhIGNsYXNzPSJtb3ot dHh0LWxpbmstYWJicmV2aWF0ZWQiIGhyZWY9Im1haWx0bzpkZXRuZXRAaWV0Zi5vcmciPmRldG5l dEBpZXRmLm9yZzwvYT48YnIgY2xhc3M9IiI+DQo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0 ZXh0IiBocmVmPSJodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RldG5ldCI+ aHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9kZXRuZXQ8L2E+PGJyIGNsYXNz PSIiPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2 Pg0KPGJyIGNsYXNzPSIiPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0K PC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPC9i b2R5Pg0KPC9odG1sPg0K --_000_6D2EFDE8452C40C7B4AF5300B995FE2Bepflch_-- From nobody Fri Mar 26 04:53:54 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5EBC43A1C84 for ; Fri, 26 Mar 2021 04:53:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.351 X-Spam-Level: X-Spam-Status: No, score=-2.351 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.251, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OTYnJ6vXRHBr for ; Fri, 26 Mar 2021 04:53:47 -0700 (PDT) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80059.outbound.protection.outlook.com [40.107.8.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 501A23A1C83 for ; Fri, 26 Mar 2021 04:53:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dFcICe1YIL5pNT25g1wXurs1j6aK2OsICHpXqiz6wE/hL//V4esfBAtPBpBV3Oju6epmM+z1aVYplmk+I+QQXB8ZZtjGtY1Gsr6lhTwetcfp/b60CrdCEtjDzkWHw8qtzMl8VGIIbD3wRR2JYAzOcduwqNV+TMLyJRoFV5fjzOxjhcOMeF9oEIxXXDn0PRMa198hyfo7InyDL/gjsY8lWVHOO+wSiwdA4dKjND0Uf4Lac+Muc35JwYNauAG/bswAeiI5gx8boIENsS49C5bAz6KYHVeURK6UHRTa2VeQ+sT8dVLIYLfdVt+C1sfT5uRqiFOtnnde1zEA7mHwgIL4OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mY6z8Z/dju8SkrRzaboxFlE40bm6/nr39KcjSSrf1NM=; b=U+YZfK4Voxna+CmhifnjJYpPITrqADOVqKEpVhtA2h7rCCe4YneH8aPPm5B9A1WOZ2yykFArt7sizKcWRxE4uklhCdOw5/8iz8MFoleJsDOasTPwPs28MSTxLBOG8BjwuPFP6w78XV6J2OeoI3T+qTFvyWUXAvzWPerY8s7d35scwPc4s0BG5/lxFciEu37tKIpMkO4/3B4u9RMJo3HkheanlHwU3dYlalm9x3PInyBB+S4BW+jMrrz5NAZo69L+WcXKdfveMg2s4VkW8u9Ke8zQr8+cM5gfpMx0TIzLBbiYTm9ajEOWw9TByDOshrQdhI5AXGngXHnAVJjlX0vJuw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mY6z8Z/dju8SkrRzaboxFlE40bm6/nr39KcjSSrf1NM=; b=LDPQMb7TTLmINTM7oI7GRf+eNiwsdQHEkVKGMuiK5THYvl9sEjztjJvC3pB5NuBHpSKcDwaVHdQlQWOt5jiyTyrLDUTEvfLESCFR0F0lfTaQJYmFVkwh2JmgdmFp/wl2JIbKTb3NlIbmHFqgN5mEYWvhNgrWwUJKDKGCHaIH5jA= Received: from AM9PR07MB7204.eurprd07.prod.outlook.com (2603:10a6:20b:2cb::15) by AM9PR07MB7681.eurprd07.prod.outlook.com (2603:10a6:20b:2c2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.16; Fri, 26 Mar 2021 11:53:44 +0000 Received: from AM9PR07MB7204.eurprd07.prod.outlook.com ([fe80::60f8:2297:7367:cf31]) by AM9PR07MB7204.eurprd07.prod.outlook.com ([fe80::60f8:2297:7367:cf31%6]) with mapi id 15.20.3999.015; Fri, 26 Mar 2021 11:53:44 +0000 From: Janos Farkas To: DetNet WG Thread-Topic: Regarding IPR on draft-tpmb-detnet-oam-framework-00 Thread-Index: AdciNqRMRQOaumonSXmW7e3AkjK0qw== Date: Fri, 26 Mar 2021 11:53:44 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [2a02:ab88:36c1:cb00:a4e8:50cd:c934:6f0d] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 813d809c-92c0-41f3-d033-08d8f04dcee7 x-ms-traffictypediagnostic: AM9PR07MB7681: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jI5h4IJscnFuVQ9lfCAaT+08XldKnVnwKoRgyvNGuMxqGbZMQXULNPWlcIil6c3S0x7JLQAxmL7aYNMtDVVCKlN2/CDipz3ILezTGk+Vn7As97GzPGu0BrjO4vvA1dcIuPO4YRLHRwCL2OEFwg69059hNzzTh2tZubDdoApCHH4OFri+frkPjp0aRpjtARqpUop2eG/bCpSpRRbTTYiyn82VMFhWCMRcYKPsrCVtnOCrsmz0AoJX9/wT9a7+ov57XAL3yUGANlv0zFxpWnzV3LLv5KCh8Ky8huDzH0HW+3s/WCF7jGCxLXElhKG6hg4DFzQGDKSLUKu5CPlKLKOv55NQ3bx/lB0KNeY7imuBbeUKLvgaGi6Ay/AienI82wXvIn7k2sCSOxNPOmgmzneZ4cxFGLUEZfFS7KqyEUBWazf7cqn+cCILGYFBzTYfhfEuUHWmUbOry66F1a8KlBZ56i3ZHYutd0wX+q9rnhy57FBukGRvBp5cBlnKXxdB6RvSJx5ocyt7wcmm/Gg1TId55nh7yIk1PYScw6gFQMpUGgJ5yPusAsp3mFC3bTYPZ65IblUxcNEaUXGHRQhofWffr7ZGlIxZ5ciT+vKQR5T91AIX2A7wpIZZ6FSIjn4ZFdW3j6CSH/XlPw6uW99bCgjfRXLsi/TE4otygs+qmhnNHvGYWYQX3LJz8sfjR1LjrJD4hRj5Pet38rREDRttjNSkCD4ZrQoN4Tr57iIe+nBbBDgFK58H1PNRBSkVw6PeyR63 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM9PR07MB7204.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39860400002)(376002)(346002)(366004)(136003)(316002)(71200400001)(52536014)(186003)(38100700001)(64756008)(66946007)(6506007)(5660300002)(55016002)(6916009)(166002)(966005)(8936002)(83380400001)(2906002)(478600001)(9686003)(33656002)(7696005)(76116006)(66446008)(66556008)(86362001)(66476007)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?NuHXGsdj1d7kZyG1G06KSgV+mapQj86UPP0ShO+zxgWH5j/8H54fpk5B2F?= =?iso-8859-1?Q?HXwwauXGjfpGW4/BZEnf8zWU24zfmDk1OQldCfvGtIx0VmWRPw/5yBrE+I?= =?iso-8859-1?Q?lWxMP3eS6DjMC8oc3vpyEw8AsVEKHi3Ivh0w3QBsxkEscAm7XUXmtuBjmK?= =?iso-8859-1?Q?XWSG7MEHTQ7Ypk1pbvWu2xhM4MX/5+6yrDm/Qm5kz/V39O6fCXLKGHIY2m?= =?iso-8859-1?Q?3ktrCFGd8S3bisp3I6rdLtInSTBJyY409Ig7L7pPKq0TP7EiDRq1+jkI+g?= =?iso-8859-1?Q?5uecqC8vbDJAFchFRCwqvcBMjcTYDiZMDKwQqDuvFAnYyC12f6ay0AC/3n?= =?iso-8859-1?Q?0OQgXQ97BY2fhAj94i1Llwdg3po/O1Wx287T9r4OoNg1WTB73fEA/AXrJ9?= =?iso-8859-1?Q?PET7zACk8+4HXxkGqaZYYiBijZp1VQAq3ZC80Pg1O5XHQPN0oXCncEb19z?= =?iso-8859-1?Q?1wTepfptqWW1rXlGu3IzYxA6MXWmvJ1UcdkIpM6TZt8zYyWFOfODFxo9cG?= =?iso-8859-1?Q?Xir5xjPtGMUeWIAUjISXFcwT2nY0ElIrOceeWFjUjVEKwfBoiIo9cqxm7n?= =?iso-8859-1?Q?emxnmnRkMymLlyyiJq74G/VSw5ZGze6vAB5UNTfNxVh47EIsmZGh6qkj1T?= =?iso-8859-1?Q?2RMbjn/S5AzbCon1GcMkpzKyZdi+vFskY1G7XgxA1zpEuJM6DeCmMVg86g?= =?iso-8859-1?Q?nni5WZrryBB4a3ZARY3/OdaQ7Qxv97dNR+DZ6A+8BOJxTL8qJdro1IzEc9?= =?iso-8859-1?Q?43tWp0qEHP0gMhKB+qsI465eNa9NVa0FDcZUk+WME6tnriEY3scEKfTuan?= =?iso-8859-1?Q?dtFTzTfWITN9InQJddgoG1/RYYmIcqMJeXeEk+bP3+gPDSWfYejMpjKjto?= =?iso-8859-1?Q?9a1ikS6FZVH019DufcR99iPnRvVGjiNFzoWnj73pLmYzqEPq78JHmam3cl?= =?iso-8859-1?Q?pz3MbHiLXvezgC2Jl1ubavpeZymvo0YdoGZkfBCg0+xcrSNY7tcsHAnAxH?= =?iso-8859-1?Q?M0yckYzFU/vhVa/LuXnCUmS00oq+II2ZNRkfeH/phQaW8AOjCyG1xh9Sk2?= =?iso-8859-1?Q?tvJdhdaHPunYBB4X6OItxIj1kYaEQ4ZFwa98nRL5MqE64qflicoT9hYKGC?= =?iso-8859-1?Q?nmuZUnXPZxUb7B14A9CrtL/RYIsMm/12G7Gk5xQk/6nAlDHi4FpvGtJEme?= =?iso-8859-1?Q?0IyqEus2DVJZeDl9Zv0KvYePnAZ7l6mhZFsoQv0L/DN9NghGPCSBQLbSwm?= =?iso-8859-1?Q?hj2mj/cl4IPeZwGYyKs23sfWWdwhowtt7NT21DaK0A0rDK/GMyU5PzgJmR?= =?iso-8859-1?Q?RZrGn4Tsqwot/yoSG5MNieNdVb28U4/sRAc/ISh0hM6d9G3hXkQss2tMgQ?= =?iso-8859-1?Q?AOFeKypB5QmXM6H1iImaivc1HGfREFgxfiGe4qohNBUzTLH501EVJlxFa7?= =?iso-8859-1?Q?ZHNeQxD+M8nkJ0Ei?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM9PR07MB7204B92ABFE37047B57B28C1F2619AM9PR07MB7204eurp_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM9PR07MB7204.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 813d809c-92c0-41f3-d033-08d8f04dcee7 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Mar 2021 11:53:44.5908 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xwZTVND2RWvSyXFwUVTo4IGyRE4SLXidApF5/wcUalP+j5gwkfrHLZdjMeuC9Zd7aJlNLfQyle4NHiMJgFId5jGmxI/f9SVRBox0zAvjNi0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7681 Archived-At: Subject: [Detnet] Regarding IPR on draft-tpmb-detnet-oam-framework-00 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2021 11:53:52 -0000 --_000_AM9PR07MB7204B92ABFE37047B57B28C1F2619AM9PR07MB7204eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Authors, Contributors, WG, As part of the preparation for polling for WG document adoption: Are you aware of any IPR that applies to draft identified above? Please state either: "No, I'm not aware of any IPR that applies to this draft" or "Yes, I'm aware of IPR that applies to this draft" If so, has this IPR been disclosed in compliance with IETF IPR rules (see RFCs 3979, 4879, 3669 and 5378 for more details)? If yes to the above, please state either: "Yes, the IPR has been disclosed in compliance with IETF IPR rules" or "No, the IPR has not been disclosed" If you answer no, please provide any additional details you think appropriate. If you are listed as a document author or contributor please answer the above by responding to this email regardless of whether or not you are aware of any relevant IPR. This document will not advance to the next stage until a response has been received from each author and listed contributor. NOTE: THIS APPLIES TO ALL OF YOU LISTED IN THIS MESSAGE'S TO LINES. If you are on the WG email list or attend WG meetings but are not listed as an author or contributor, we remind you of your obligations under the IETF IPR rules which encourages you to notify the IETF if you are aware of IPR of others on an IETF contribution, or to refrain from participating in any contribution or discussion related to your undisclosed IPR. For more information, please see the RFCs listed above and http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualProperty. Thank you, J=E1nos (and Lou) --_000_AM9PR07MB7204B92ABFE37047B57B28C1F2619AM9PR07MB7204eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Authors, Contributors, WG,

As part of the preparation for polling for WG docume= nt adoption:

Are you aware of any IPR that applies to draft ident= ified above?

    Please state either:

    "No, I'm not aware of any IP= R that applies to this draft"

    or

    "Yes, I'm aware of IPR that = applies to this draft"

If so, has this IPR been disclosed in compliance wit= h IETF IPR rules

(see RFCs 3979, 4879, 3669 and 5378 for more details= )?

If yes to the above, please state either:=

    "Yes, the IPR has been discl= osed in compliance with IETF IPR rules"

    or

    "No, the IPR has not been di= sclosed"

If you answer no, please provide any additional deta= ils you think

    appropriate.

If you are listed as a document author or contributo= r please answer the

above by responding to this email regardless of whet= her or not you are

aware of any relevant IPR. This document will = not advance to the next

stage until a response has been received from each a= uthor and listed

contributor. NOTE: THIS APPLIES TO ALL OF YOU = LISTED IN THIS

MESSAGE'S TO LINES.

If you are on the WG email list or attend WG meeting= s but are not listed

as an author or contributor, we remind you of your o= bligations under

the IETF IPR rules which encourages you to notify th= e IETF if you are

aware of IPR of others on an IETF contribution, or t= o refrain from

participating in any contribution or discussion rela= ted to your

undisclosed IPR. For more information, please see th= e RFCs listed above

and http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualProperty.

Thank you,
J=E1nos (and Lou)

--_000_AM9PR07MB7204B92ABFE37047B57B28C1F2619AM9PR07MB7204eurp_-- From nobody Fri Mar 26 04:58:44 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46EB03A1CB3 for ; Fri, 26 Mar 2021 04:58:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=it.uc3m.es Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2XbMYXV5WLVM for ; Fri, 26 Mar 2021 04:58:38 -0700 (PDT) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D91CF3A1CB0 for ; Fri, 26 Mar 2021 04:58:37 -0700 (PDT) Received: by mail-ed1-x52f.google.com with SMTP id l18so5986263edc.9 for ; Fri, 26 Mar 2021 04:58:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=it.uc3m.es; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qh7Qd70Anz1X1gNDt99Vh173/t/G0DWeCODl4fNZ8OE=; b=aqAKnlVijZ1EN2vqrgxv4mThX6VcWULygbttS+joHUA7vhBzp56JIZrzwXMUjA0ahq hpIIK6dtWhfWLmlrBkvLimTwJSPPKalIDr1WgiU6WxKU5K0fVCKJTzDp196Abj07y5cU Rm+2Ql3jzugKMBykk0LrwMTKr4ugAj/HbhDhLm6eLZ0wW1OXq22IWwtpaqzRsXAOT2Xu DFY6kZueYNZziim57GZOPL7Jwwziii+F8laIU9ADYjUyE8e7vk16fOe3+f3FjE4JXhAO x67Q6MMFBzMj4yhVk/RcFoe9rokc9EhbgmoLYwtMDVEvOGq+hEh8QEBWg01U/eDmZi8A O3fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qh7Qd70Anz1X1gNDt99Vh173/t/G0DWeCODl4fNZ8OE=; b=TKzEoY9mqj7PQoGzw2uE6hQFqEOLKCLzwtFk/DkVfhungLRluoeP5nrBI9yxCxAOXX 5FV6kTmb2c++Sr6K22SoO4pTNWwjcWh70WLZdbVKnoZZAprtadNBp5CJ2OegQ+JrNF98 ZIrVZnov85N80e20Wk08OyEn0J7yxv8TJYWk/QM7gzKjDsjOI6RkD4DlIAbDgPLXuJya L7KQDhmh9/6TLq8YY09DtJjK9NSckIm+Dg4QU+dwDpzExz279oD1apPsMcThDnrS5UJj 5bSSKwiKPveoLOKMkGpeGjGBKMhIpTZrIeqBgoWoKOQSRp0AF4F8V7FyPK5JTbkah06z FI0Q== X-Gm-Message-State: AOAM530JZxIZQjmPzBVTyNACBGW89ZECFgTbM0JCS5HFA4mPaV0XmdTf aDgdMFIv7eVc2ZXT5sUd5dFxaHBcOeG3Nj+QelxgNfI4hyEvFWyS X-Google-Smtp-Source: ABdhPJy53eW2pkGJRnOZNRfwpA+PrkPveRH9Y20j3BAVssf3hYpm+C5EWvvm5qTo8qV89A0jYzxlPn3uyRTsPRcGlBs= X-Received: by 2002:a05:6402:354b:: with SMTP id f11mr2562232edd.361.1616759914450; Fri, 26 Mar 2021 04:58:34 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: CARLOS JESUS BERNARDOS CANO Date: Fri, 26 Mar 2021 12:58:18 +0100 Message-ID: To: Janos Farkas Cc: DetNet WG Content-Type: multipart/alternative; boundary="00000000000063691305be6f4099" Archived-At: Subject: Re: [Detnet] Regarding IPR on draft-tpmb-detnet-oam-framework-00 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2021 11:58:43 -0000 --00000000000063691305be6f4099 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, No, I'm not aware of any IPR that applies to this draft. Thanks, Carlos On Fri, Mar 26, 2021 at 12:54 PM Janos Farkas wrote: > Authors, Contributors, WG, > > > > As part of the preparation for polling for WG document adoption: > > > > Are you aware of any IPR that applies to draft identified above? > > > > Please state either: > > > > "No, I'm not aware of any IPR that applies to this draft" > > or > > "Yes, I'm aware of IPR that applies to this draft" > > > > If so, has this IPR been disclosed in compliance with IETF IPR rules > > (see RFCs 3979, 4879, 3669 and 5378 for more details)? > > > > If yes to the above, please state either: > > > > "Yes, the IPR has been disclosed in compliance with IETF IPR rules" > > or > > "No, the IPR has not been disclosed" > > > > If you answer no, please provide any additional details you think > > appropriate. > > > > If you are listed as a document author or contributor please answer the > > above by responding to this email regardless of whether or not you are > > aware of any relevant IPR. This document will not advance to the next > > stage until a response has been received from each author and listed > > contributor. NOTE: THIS APPLIES TO ALL OF YOU LISTED IN THIS > > MESSAGE'S TO LINES. > > > > If you are on the WG email list or attend WG meetings but are not listed > > as an author or contributor, we remind you of your obligations under > > the IETF IPR rules which encourages you to notify the IETF if you are > > aware of IPR of others on an IETF contribution, or to refrain from > > participating in any contribution or discussion related to your > > undisclosed IPR. For more information, please see the RFCs listed above > > and http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualProperty. > > > > Thank you, > J=C3=A1nos (and Lou) > > > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://www.ietf.org/mailman/listinfo/detnet > --00000000000063691305be6f4099 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

No, I'm not aware of any IPR th= at applies to this draft.

Thanks,
Carlos

On Fri, Mar 26, 2021 at 12:54 PM Janos Farkas <= Janos.Farkas=3D40ericsson.= com@dmarc.ietf.org> wrote:

Authors, Contributors, WG,

=C2=A0

As part of the preparation for polling for WG docume= nt adoption:

=C2=A0

Are you aware of any IPR that applies to draft ident= ified above?

=C2=A0

=C2=A0=C2=A0=C2=A0 Please state either:

=C2=A0

=C2=A0=C2=A0=C2=A0 "No, I'm not aware of an= y IPR that applies to this draft"

=C2=A0=C2=A0=C2=A0 or

=C2=A0=C2=A0=C2=A0 "Yes, I'm aware of IPR t= hat applies to this draft"

=C2=A0

If so, has this IPR been disclosed in compliance wit= h IETF IPR rules

(see RFCs 3979, 4879, 3669 and 5378 for more details= )?

=C2=A0

If yes to the above, please state either:<= /u>

=C2=A0

=C2=A0=C2=A0=C2=A0 "Yes, the IPR has been discl= osed in compliance with IETF IPR rules"

=C2=A0=C2=A0=C2=A0 or

=C2=A0=C2=A0=C2=A0 "No, the IPR has not been di= sclosed"

=C2=A0

If you answer no, please provide any additional deta= ils you think

=C2=A0=C2=A0=C2=A0 appropriate.

=C2=A0

If you are listed as a document author or contributo= r please answer the

above by responding to this email regardless of whet= her or not you are

aware of any relevant IPR.=C2=A0 This document will = not advance to the next

stage until a response has been received from each a= uthor and listed

contributor.=C2=A0 NOTE: THIS APPLIES TO ALL OF YOU = LISTED IN THIS

MESSAGE'S TO LINES.

=C2=A0

If you are on the WG email list or attend WG meeting= s but are not listed

as an author or contributor, we remind you of your o= bligations under

the IETF IPR rules which encourages you to notify th= e IETF if you are

aware of IPR of others on an IETF contribution, or t= o refrain from

participating in any contribution or discussion rela= ted to your

undisclosed IPR. For more information, please see th= e RFCs listed above

and http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualProperty.

=C2=A0

Thank you,
J=C3=A1nos (and Lou)

=C2=A0

_______________________________________________
detnet mailing list
detnet@ietf.org https://www.ietf.org/mailman/listinfo/detnet

--00000000000063691305be6f4099-- From nobody Fri Mar 26 05:30:13 2021 Return-Path: X-Original-To: detnet@ietfa.amsl.com Delivered-To: detnet@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E8CAB3A1D6B for ; Fri, 26 Mar 2021 05:30:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.272 X-Spam-Level: X-Spam-Status: No, score=0.272 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_HELO_IP_MISMATCH=2.368, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=unistra.fr Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdUHCfulBzEj for ; Fri, 26 Mar 2021 05:30:06 -0700 (PDT) Received: from smtpout01-ext1.partage.renater.fr (smtpout01-ext1.partage.renater.fr [194.254.240.32]) by ietfa.amsl.com (Postfix) with ESMTP id 08A4A3A1D6C for ; Fri, 26 Mar 2021 05:30:05 -0700 (PDT) Received: from zmtaauth01.partage.renater.fr (zmtaauth01.partage.renater.fr [194.254.240.25]) by smtpout10.partage.renater.fr (Postfix) with ESMTP id CCE8462C5A; Fri, 26 Mar 2021 13:30:00 +0100 (CET) Received: from zmtaauth01.partage.renater.fr (localhost [127.0.0.1]) by zmtaauth01.partage.renater.fr (Postfix) with ESMTPS id C069D140716; Fri, 26 Mar 2021 13:30:00 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zmtaauth01.partage.renater.fr (Postfix) with ESMTP id AD59414073E; Fri, 26 Mar 2021 13:30:00 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.10.3 zmtaauth01.partage.renater.fr AD59414073E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unistra.fr; s=CF279DD4-6F58-4C59-BB33-73FDC6DFC1E3; t=1616761800; bh=yCfZ44NvuHs11pR9mcdtcgjsHDh2dnhISGg0AwUd304=; h=From:Message-Id:Mime-Version:Date:To; b=HuIlXPMZ1y4HbF5YDwCN44lEpsQ12EhAobnVMSyuSq0vYGUTTciUuYq9fsxOUx5gh pNBw0qG2eUraj7IJnJUuH9DOgohDZNBltbaEolN0rCb6qVL79OrITFpWC7X3Y950dk ognvoP1MW9bILn3eSVmvRxoeH831x4zwlXluVmHxxGHn0pEziBi/N/aWVyD/APtpIB +Taqf/q5rhHhNsgfeGLFYpkjMA5PANIpdSPZW2oSCc1DQuFtaACMK6TREOzMeZiZlY 9PRRJiFXoH6xNJRxR7XimZdP9gNUg2T8KrxYau0qHNZxzTWwhi7r4gJGqp87Q9SRLv 6KmWjKtTCpZhg== X-Virus-Scanned: amavisd-new at zmtaauth01.partage.renater.fr Received: from zmtaauth01.partage.renater.fr ([127.0.0.1]) by localhost (zmtaauth01.partage.renater.fr [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7FuqFEuMBSzO; Fri, 26 Mar 2021 13:30:00 +0100 (CET) Received: from 130.79.91.198 (unknown [194.254.241.251]) by zmtaauth01.partage.renater.fr (Postfix) with ESMTPA id 6C036140723; Fri, 26 Mar 2021 13:30:00 +0100 (CET) From: Fabrice Theoleyre Message-Id: <7EA9B7AD-10EA-4E45-9DA7-794AAA0A49D4@unistra.fr> Content-Type: multipart/signed; boundary="Apple-Mail=_115210E7-B196-4AFC-851C-82D98849428B"; protocol="application/pkcs7-signature"; micalg=sha-256 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Date: Fri, 26 Mar 2021 13:30:00 +0100 In-Reply-To: Cc: DetNet WG To: Janos Farkas References: X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Renater-Ptge-SpamState: clean X-Renater-Ptge-SpamScore: -100 X-Renater-Ptge-SpamCause: gggruggvucftvghtrhhoucdtuddrgeduledrudehvddggedvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecutffgpfetvffgtfenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffktgggufffjgfvfhfosehgtdhmrehhtdejnecuhfhrohhmpefhrggsrhhitggvucfvhhgvohhlvgihrhgvuceothhhvgholhgvhihrvgesuhhnihhsthhrrgdrfhhrqeenucggtffrrghtthgvrhhnpeejleegkeeukeevkeeihfethefhieegheduudeggfejjedvleejleehheeugfdvhfenucffohhmrghinhepihgvthhfrdhorhhgnecukfhppeduleegrddvheegrddvgedurddvhedunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelgedrvdehgedrvdeguddrvdehuddphhgvlhhopedufedtrdejledrledurdduleekpdhmrghilhhfrhhomhephfgrsghrihgtvgcuvfhhvgholhgvhihrvgcuoehthhgvohhlvgihrhgvsehunhhishhtrhgrrdhfrheqpdhrtghpthhtoheplfgrnhhoshdrhfgrrhhkrghspeegtdgvrhhitghsshhonhdrtghomhesughmrghrtgdrihgvthhfrdhorhhgpdhrtghpthhtohepuggvthhnvghtsehivghtfhdrohhrgh Archived-At: Subject: Re: [Detnet] Regarding IPR on draft-tpmb-detnet-oam-framework-00 X-BeenThere: detnet@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions on Deterministic Networking BoF and Proposed WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2021 12:30:11 -0000 --Apple-Mail=_115210E7-B196-4AFC-851C-82D98849428B Content-Type: multipart/alternative; boundary="Apple-Mail=_6C014BAD-9394-4822-8747-BCD43249E52E" --Apple-Mail=_6C014BAD-9394-4822-8747-BCD43249E52E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Dear Janos, all, No, I'm not aware of any IPR that applies to this draft. Cheers, Fabrice > Le 26 mars 2021 =C3=A0 12:53, Janos Farkas = a =C3=A9crit : >=20 > Authors, Contributors, WG, > =20 > As part of the preparation for polling for WG document adoption: > =20 > Are you aware of any IPR that applies to draft identified above? > =20 > Please state either: > =20 > "No, I'm not aware of any IPR that applies to this draft" > or > "Yes, I'm aware of IPR that applies to this draft" > =20 > If so, has this IPR been disclosed in compliance with IETF IPR rules > (see RFCs 3979, 4879, 3669 and 5378 for more details)? > =20 > If yes to the above, please state either: > =20 > "Yes, the IPR has been disclosed in compliance with IETF IPR = rules" > or > "No, the IPR has not been disclosed" > =20 > If you answer no, please provide any additional details you think > appropriate. > =20 > If you are listed as a document author or contributor please answer = the > above by responding to this email regardless of whether or not you are > aware of any relevant IPR. This document will not advance to the next > stage until a response has been received from each author and listed > contributor. NOTE: THIS APPLIES TO ALL OF YOU LISTED IN THIS > MESSAGE'S TO LINES. > =20 > If you are on the WG email list or attend WG meetings but are not = listed > as an author or contributor, we remind you of your obligations under > the IETF IPR rules which encourages you to notify the IETF if you are > aware of IPR of others on an IETF contribution, or to refrain from > participating in any contribution or discussion related to your > undisclosed IPR. For more information, please see the RFCs listed = above > and = http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualProperty = . > =20 > Thank you, > J=C3=A1nos (and Lou) > =20 > _______________________________________________ > detnet mailing list > detnet@ietf.org > https://www.ietf.org/mailman/listinfo/detnet = --Apple-Mail=_6C014BAD-9394-4822-8747-BCD43249E52E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Dear = Janos, all,

No, = I'm not aware of any IPR that applies to this draft.

Cheers,
Fabrice

Le 26 = mars 2021 =C3=A0 12:53, Janos Farkas <Janos.Farkas=3D40ericsson.com@dmarc.ietf.org> a =C3=A9cr= it :

Authors, Contributors, WG,

As part of the preparation = for polling for WG document adoption:

Are you aware of any IPR that applies to draft = identified above?

    Please = state either:

    "No, = I'm not aware of any IPR that applies to this draft"
    or
    "Yes, = I'm aware of IPR that applies to this draft"

If so, has this IPR been = disclosed in compliance with IETF IPR rules
(see RFCs 3979, 4879, 3669 = and 5378 for more details)?

If yes to the above, please state either:

    "Yes, = the IPR has been disclosed in compliance with IETF IPR rules"
    or
    "No, = the IPR has not been disclosed"

If you answer no, please provide any additional = details you think
    appropriate.

If you are listed as a document author or = contributor please answer the
above by responding to this email regardless of = whether or not you are
aware = of any relevant IPR. This document will not advance to the = next
stage until a = response has been received from each author and listed
contributor. NOTE: = THIS APPLIES TO ALL OF YOU LISTED IN THIS
MESSAGE'S TO LINES.

If you are on the WG email = list or attend WG meetings but are not listed
as an author or = contributor, we remind you of your obligations under
the IETF IPR rules which = encourages you to notify the IETF if you are
aware of IPR of others on = an IETF contribution, or to refrain from
participating in any contribution or discussion = related to your
undisclosed= IPR. For more information, please see the RFCs listed above
and http://trac.tools.ietf.org/group/iesg/trac/wiki/IntellectualPro= perty.

Thank you,
J=C3=A1nos (and Lou)

_______________________________________________
detnet mailing list
detnet@ietf.org
https://www.ietf.org/mailman/listinfo/detnet