Object-based pNFS Operations

- moderator - Shepler

- = reader -=20 Eisler

- scribe - Khan

- author - Noveck

- inspectors: Erasani, Gordon, = Kustarz

In this=20 meeting, we covered lines 8751 -=20 9446, 9530 - 9728, 10476 - 10519 = from=20 draft 13.

I am listing=20 the review comments received during the review by name, line number and=20 comment.

lines 8751=20 - = 9446

Khan - L 8775 - It should be emphasized that = the server=20 must not allow reclaims to happen if it is not storing the lock = information on=20 stable storage in order to be spec compliant.

Kustarz - L 8781 - Clarify what 'false positives' = means=20 here.

Noveck - L 8826 - Another case missing here is the = one for a=20 persistent session where the state is lost and the client is asked to = reclaim=20 its state.

Kustarz - General Comment - All state related = errors should be=20 consolidated and explained together, maybe in a separate=20 section.

Eisler - L 8834 - s/as discussed above/as = discussed in section=20 blah.

Eisler - L 8838 - These lines are not correct and = need to=20 clarify what some of the SEQUENCE flags do.

Noveck - General comment - The two edge conditions = mentioned=20 in 8.4.3 are only in terms of lease expiration and not in the context of = server=20 revocation of locks. That needs to be addressed either in this section = or this=20 section needs to be somehow part of 8.4.3.

Ersani/Eisler - General comment - In the = desription of=20 test_stateid, it should be mentioned that the client may do this op = whenever it=20 wants, but it should definitely do this when any locks have been revoked = and the=20 server should allow the client to issue this op multiple=20 times.

Eisler - L 8899, 8917 -=20 s/lock/lease

Eisler - L 8978 - server should ignore these fields and = not return=20 an error.

Noveck - L 8971 - Mention here that the cleintid in = CREATE_SESSION and=20 DESTROY_SESSION is not=20 ignored.

Eisler - L 8984 - OPEN and CREATE should be in lower=20 case.

&= nbsp;

Eisler - L 8994 - s/lock/byte range=20 lock

<= /SPAN>

Khan/Kustarz - L 9004 - byte range locks should either be = referred to as=20 byte range locks or record locks in all places, currently this is not=20 consistent. Same for=20 octet locks.=

<= /SPAN>

Noveck - L 9029 - This should be mentioned in the previous=20 chapter.

<= /SPAN>

Kustarz - General Comment - Clarify all references to owners to = be=20 specifically lock owner/open owner or state=20 owner.

<= /SPAN>

Eisler - L 9051 - special stateid section should be referenced=20 here.

<= /SPAN>

Noveck - L 9132 -=20 s/OPEN/stateid

<= /SPAN>

Gordon - Comment for this section - replace setattr for set = size with=20 write.

<= /SPAN>

Eisler - L 9176 - remove the=20 line.

<= /SPAN>

Eisler - L 9157 - There is no difference between using special = stateids=20 or regular stateids for this=20 scenario.<= /SPAN>

<= /SPAN>

Eisler - L 9254 - This is not true, it should be clarified what = the new=20 behavior=20 is.=

<= /SPAN>=

Eisler - L 9318 - Needs=20 clarification.

<= /SPAN>=

Eisler - L 9381 - Section on stateid creation should be = referenced and=20 explained what happens to the old stateid in this=20 case.<= /SPAN>=

<= /SPAN>=

Eisler - L 9403 -=20 s/open/OPENs<= /SPAN>=

<= /SPAN>= &n= bsp;

Kustarz - L 9403 - reference to the section that discusses how = parallel=20 opens should be handled and=20 implemented.<= /SPAN>=

<= /SPAN>

Eisler - L 9413 -=20 s/ay/may

<= /SPAN>

Eisler - L 9417 - inadvertently=20 (typo)=

<= /SPAN>

Noveck - L 9435 - This needs to be discussed on the mailing = list. There=20 could be issues here with part of the request being processed by = one server=20 instance and part of it by another=20 one.

<= /SPAN>=

lines 9530 -=20 9728=

Eisler - L 9554 - s/callback/back=20 channel

<= /SPAN>=

Khan - L 9581 - Clarify that nfs4err_delay could be returned in = this case=20 as=20 well.<= /SPAN>

<= /SPAN>=

Eisler - L 9620 -=20 s/leases/lease

<= /SPAN>=

Khan - L 9651 -=20 s/but/and<= /SPAN>=

<= /SPAN>=

Eisler - L 9693 - s/reclaim a/reclaim via=20 a<= /DIV>

<= /SPAN>=

Eisler - L 9701 - remove=20 quotes.<= /SPAN>

<= /SPAN>= &n= bsp;

Gordon - L 9648 - CLAIM_DELEGATE_PREV_FH should be mentioned = here as=20 well.<= /SPAN>=

<= /SPAN>= &n= bsp;

Eisler - L 9705 -=20 s/freeing/revocation= <= /SPAN>=

<= /SPAN>= <= /SPAN>

Eisler - L 9723 - Needs to be clarified if the state = needs to be in=20 stable storage, the proper section should be referenced=20 here.<= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>=

lines 10476 - 10519=20 <= /SPAN>

Noveck - L 10510 - Needs = to be=20 clarified.=

Khan - L 10516 -=20 s/leases/lease

Eisler - General = comment - It=20 should be mentioned somewhere that the client needs to look at the = status flags=20 returned from SEQUENCE because it might need to do some actions based on = what it=20 got.<= /SPAN>

Gordon - L 10504 - s/a reasonable = time/atleast one=20 lease=20 period=

Khan - L 10534 - It should be added here that = any=20 open/lock state subsumed by the delegation does not get revoked when the = delegation gets=20 revoked.<= /SPAN>

Kustarz - General = comment - It=20 should be mentioned somewhere that when doing IO the order of checking = stateids=20 should be=20 delegation/lock/open/special.<= /SPAN>= <= /SPAN>

= <= /SPAN>=

thanks.= <= /SPAN>=

Saadia<= /SPAN>= <= /SPAN>=

= <= /SPAN>=

------_=_NextPart_001_01C7EA98.AA445159-- --===============1997227766== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 --===============1997227766==-- From nfsv4-bounces@ietf.org Tue Sep 04 21:06:12 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISjJf-0002YE-Jh; Tue, 04 Sep 2007 21:04:23 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISiEi-00061o-IH for nfsv4@ietf.org; Tue, 04 Sep 2007 19:55:12 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISiEc-0000Qa-Lq for nfsv4@ietf.org; Tue, 04 Sep 2007 19:55:12 -0400 X-IronPort-AV: E=Sophos;i="4.20,208,1186383600"; d="scan'208,217";a="100246902" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 04 Sep 2007 16:54:50 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l84NsnPm021540 for ; Tue, 4 Sep 2007 16:54:49 -0700 (PDT) Received: from exsvlrb02.hq.netapp.com ([10.56.8.63]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 4 Sep 2007 16:54:49 -0700 Received: from exsvl02.hq.netapp.com ([10.56.8.60]) by exsvlrb02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 4 Sep 2007 16:54:49 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Tue, 4 Sep 2007 16:54:12 -0700 Message-ID: <044B81DE141D7443BCE91E8F44B3C1E206EA1A37@exsvl02.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Review comments for Locking and Directory DelegationsReview # 3 Thread-Index: AcfvTuTN7psmU7bOTp2ReUMoPBh6TQ== From: "Khan, Saadia" To: X-OriginalArrivalTime: 04 Sep 2007 23:54:49.0037 (UTC) FILETIME=[FA5F0FD0:01C7EF4E] X-Spam-Score: 0.0 (/) X-Scan-Signature: f164987c086f4010951d468865aaed1a X-Mailman-Approved-At: Tue, 04 Sep 2007 21:04:17 -0400 Subject: [nfsv4] Review comments for Locking and Directory DelegationsReview # 3 X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0353770772==" Errors-To: nfsv4-bounces@ietf.org This is a multi-part message in MIME format. --===============0353770772== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C7EF4E.FA34BCBA" This is a multi-part message in MIME format. ------_=_NextPart_001_01C7EF4E.FA34BCBA Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The third review meeting for locking and directory delegations was held on September 4, 2007. =20 The roles in this meeting were: - moderator - Shepler - reader - Eisler - scribe - Khan - author - Noveck - inspectors: Erasani, Gordon, Kustarz In this meeting, we covered lines 10547 - 10560, 13332 - 13478, 24310 - 24588, 25788 - 25994 and 26044 - 26159 from draft 13. I am listing the review comments received during the review by name, line number and comment. =20 lines 10547 - 10560 =20 Eisler - L 10555 - It should be mentioned that this operation is valid for all file types except dirs. =20 Khan - L 10557 - lock state should be mentioned here as well. =20 Eisler - Also mention that callbacks can be done in case the delegation is not immediately available. =20 Ersani - Clarified that this op can be issued at reclaim time as well. =20 lines 13332 - 13478 =20 Eisler - Delegation related text should be in the caching chapter. =20 Eisler - L 13412 - s/nfsv4/nfsv4.1 =20 Eisler - L 13411 - Clarify by mentioning that the only way to avoid delegation recall is to make changes through a session associated with the clientid. =20 Eisler - L 13416 - add that this is true only if notifications are requested by the client. =20 Eisler - L 13426 - add the third case where multiple clients hold delegations and one client makes the changes. =20 Eisler - L 13452 - mention CB_RECALL here =20 Eisler - L 13456 - Remove the line starting with 'Also if .." =20 Saadia - L 13463 - Remove this paragraph. =20 Noveck - L 13477 - yes, the state should be removed. =20 lines 24310 - 24588 =20 Noveck - L 24312 - s/test/free one =20 Eisler - L 24353 - It should be mentioned in the implementation section that the client should test the stateid before freeing it. =20 Noveck - L 24336 - Add device id mapping here if there is consensus on that issue. =20 Khan - L 24341 - Divide this up into multiple sentences for clarity. =20 Eisler - L 24371 - remove (cfh) =20 Eisler - L 24386 - change to be a bitmap instead. =20 Noveck - L 24461 - remove 'also' =20 Khan - L 24467 - delete the last sentence =20 Eisler - L 24479 - remove the part about cookie verifier =20 Eisler - specific args/response fields should be referenced in this section =20 Khan - L 24493 - Lines starting from here in this paragraph should be moved to the implementation section and '..' should be changed to parent. =20 Noveck - L 24499 - Clarify that an error is not returned if a callback will be done when the delegation becomes available. Also note that if the server decides not to send a notification then it can set deleg_avail to FALSE and return DLG_UNAVIL. =20 Gordon - The server should not add bits to the supported notifications bitmask that the client did not ask for. =20 Khan - L 24504 - Also mention that if the delegation is available, the server might decide to give it to the client instead of doing a callback later. =20 Noveck - L 24499 - Look into "want" and decide if the semantics are the same as want_delegation and if so clarify that. Also look into whether this 'want' can be cancelled through the cb_wants_canelled op. =20 Khan - L 24519 - s/flags/bits =20 Eisler - L 24520 - s/direction/directory =20 Khan - L 24541 - change 'appropriately' to mention what should be set here. =20 Eisler - L 24588 - reference the section on security negotiations here. =20 Khan - L 24588 - This should be clarified and it should not say anything about setting hte cfh. =20 lines 25788 - 25994 =20 Gordon - L 25794 - s/error_codes<>/status_codes<> =20 Eisler - L 25839 - some examples and reference to the sequence op needed here. =20 Eisler - L 25844 - s/provides/returns =20 Eisler - L 25847 - s/cause processing/cause compound processing =20 Dave/Mike/Saadia - L 25855 - Add old_stateid and stale_stateid as valid errors. Also add EInval for special stateids. =20 Eilser/Kustarz - Reference the stateid generation section in the implementation section.=20 =20 Eisler - L 25865 - remove (client ID) =20 Eisler - L 25943 - It should also mention here that the server might decide to cancel the want by issuing a cb_wants_cancelled operation. =20 Khan - L 25945 - Clarification needed that in this case the server will not set the bit that tells the client that the server will callback, x-ref the open section here. =20 Eisler - L 25955 - The flags here should be indented. =20 Noveck - L 25972 - This discussion should be moved to the implementation section. It should probably say that the delegation should not be recalled in case of a conflict but its upto the server to decide. =20 Eisler - L 25978 - s/differences/difference =20 Khan - L 25988 - This combination should not be allowed and EINVAL should be returned in this case. =20 Eisler - L 25994 - This part should mention when the delegation should be recalled. =20 lines 26044 - 26159=20 =20 Khan - L 26074 - s/of/or =20 Eisler - L 26084 - s/fs/file system =20 Khan - L 26072 - It should be mentioned that the client MUST issue a reclaim_complete when its done. =20 Eisler - L 26103 - s/must/MUST =20 Eisler - L 26015/26113 - It should be mentioned here that if hte client does a non-reclaim op before doing reclaim_complete, it should be prepared for a grace error. Also even after doing a reclaim_complete, the client should be prepared for a grace error since the server might still not be out of its reclaim period. =20 Eisler - L 26116 - the two known edge conditions should be mentioned here and x-referenced. =20 thanks. Saadia =20 ------_=_NextPart_001_01C7EF4E.FA34BCBA Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

The third review meeting for locking and = directory=20 delegations was held on September = 4,=20 2007.

The=20 roles in this meeting were:

- moderator - Shepler

- reader=20 - Eisler

- scribe - Khan

- author - Noveck

- inspectors: Erasani, Gordon, = Kustarz

In this=20 meeting, we covered lines 10547 - 10560, 13332 - 13478, 24310 - 24588, = 25788 -=20 25994 and 26044 - 26159 from draft=20 13.

I am listing=20 the review comments received during the review by name, line number and=20 comment.

lines 10547 -=20 10560

Eisler - L 10555 - It should be mentioned = that this=20 operation is valid for all file types except = dirs.

Khan - L 10557 - lock state should be = mentioned here as=20 well.

Eisler - Also mention that = callbacks can=20 be done in case the delegation is not immediately=20 available.

&nbs= p;

Ersani - Clarified that this = op can be=20 issued at reclaim time as = well.

&nbs= p;

lines 13332 -=20 13478

Eisler - Delegation related = text should be=20 in the caching chapter.

&nbs= p;

Eisler - L 13412 = -=20 s/nfsv4/nfsv4.1

Eisler - L 13411 - Clarify by mentioning that = the only=20 way to avoid delegation recall is to make changes through a session = associated=20 with the=20 clientid.<= /SPAN>

Eisler - L 13416 - add that = this is true=20 only if notifications are requested by the=20 client.

Eisler - L 13426 = - add the=20 third case where multiple clients hold delegations and one client makes = the=20 changes.

Eisler - L 13452 - mention CB_RECALL=20 here

Eisler - L 13456 - Remove the line starting = with 'Also=20 if=20 .."=

Saadia - L 13463 - Remove this = paragraph.=

Noveck=20 -=20 L 13477 - yes, the state should be=20 removed.

lines 24310 -=20 24588

Noveck<= /FONT> -=20 L 24312 - s/test/free one

Eisler - L 24353 - It should = be=20 mentioned in the implementation section that the client should test = the=20 stateid before freeing = it.

&nbs= p;

Noveck -=20 L 24336 - Add device id mapping here if there is consensus on that=20 issue.

&nbs= p;

Khan - L 24341 - Divide this = up into=20 multiple sentences for = clarity.

&nbs= p;

Eisler - L 24371 = - remove=20 (cfh)

Eisler - L 24386 - change to be a bitmap=20 instead.

Noveck - L 24461 -=20 remove 'also'

Khan - L 24467 - delete the last=20 sentence

Eisler - L 24479 - remove the part about = cookie=20 verifier

Eisler - specific = args/response fields=20 should be referenced in this=20 section

Khan - L 24493 - Lines = starting from=20 here in this paragraph should be moved to the implementation section and = '..'=20 should be changed to=20 parent.

Noveck - L 24499 - = Clarify that an=20 error is not returned if a callback will be done when the delegation = becomes=20 available. Also note that if the server decides not to send a = notification then=20 it can set deleg_avail to FALSE and return=20 DLG_UNAVIL.

Gordon - The server should not = add bits to=20 the supported notifications bitmask that the client did not ask=20 for.

Khan - L 24504 - Also mention = that if the=20 delegation is available, the server might decide to give it to the = client=20 instead of doing a callback=20 later.=

Noveck - L 24499 - Look into = "want" and=20 decide if the semantics are the same as want_delegation and if so = clarify that.=20 Also look into whether this 'want' can be cancelled through the=20 cb_wants_canelled=20 op.=

Khan - L 24519 -=20 s/flags/bits<= /FONT>

Eisler - L 24520 -=20 s/direction/directory

Khan - L 24541 - change = 'appropriately' to=20 mention what should be set=20 here.<= /SPAN>

Eisler - L 24588 - reference the section on = security=20 negotiations=20 here.<= /SPAN>=

Khan - L 24588 - This should = be clarified=20 and it should not say anything about setting hte=20 cfh.<= /FONT>

lines 25788 -=20 25994

Gordon - L 25794 -=20 s/error_codes<>/status_codes<>

Eisler - L 25839 = - some=20 examples and reference to the sequence op needed=20 here.<= /SPAN>

= &nb= sp;

Eisler - L 25844 -=20 s/provides/returns<= /SPAN>

Eisler - L 25847 - s/cause=20 processing/cause compound=20 processing=

Dave/Mike/Saadia - L 25855 - = Add=20 old_stateid and stale_stateid as valid errors. Also add EInval for = special=20 stateids.<= /SPAN>=

Eilser/Kustarz - Reference the = stateid=20 generation section in the implementation section.=20 <= /FONT>

Eisler - L 25865 - remove = (client=20 ID)= <= /FONT>=

= <= /SPAN>=

Eisler - L 25943 - It should = also mention=20 here that the server might decide to cancel the want by issuing a=20 cb_wants_cancelled=20 operation.= <= /SPAN>= <= /SPAN>=

= <= /SPAN>= <= /SPAN>

Khan - L 25945 - Clarification = needed that=20 in this case the server will not set the bit that tells the client that = the=20 server will callback, x-ref the open section=20 here.<= /SPAN>= <= /SPAN>=

= <= /SPAN>= <= /SPAN>

Eisler - L 25955 - The flags = here should=20 be=20 indented.<= /SPAN>= <= /SPAN>= <= /SPAN>=

= <= /SPAN>= <= /SPAN>=

Noveck - L 25972 - This = discussion=20 should be moved to the implementation section. It should probably say = that the=20 delegation should not be recalled in case of a conflict but its upto the = server=20 to=20 decide.<= /SPAN>= <= /SPAN>= <= /SPAN>

= <= /SPAN>= <= /SPAN>=

<= /FONT>= <= /FONT>= <= /FONT>= <= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>= <= /FONT>= <= /FONT>= <= /FONT>=

<= /SPAN>= <= /FONT>= <= /FONT>= <= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>= <= /FONT>= <= /FONT>= <= /FONT>=

<= /SPAN>= <= /SPAN>= <= /SPAN>= <= /SPAN>= <= /FONT>=

<= /SPAN>= lines 26044 - 26159 =

Khan - L 26074 -=20 s/of/or

Eisler - L 26084 - s/fs/file=20 system

Khan - L 26072 - It should be mentioned that = the client=20 MUST issue a reclaim_complete when its done.

Eisler - L 26103 -=20 s/must/MUST

Eisler - L 26015/26113 - It should be = mentioned here=20 that if hte client does a non-reclaim op before doing reclaim_complete, = it=20 should be prepared for a grace error. Also even after doing a = reclaim_complete,=20 the client should be prepared for a grace error since the server might = still not=20 be out of its reclaim period.

Eisler - L 26116 - the two known edge = conditions should=20 be mentioned here and x-referenced.

thanks.

Saadia

<= /SPAN>

------_=_NextPart_001_01C7EF4E.FA34BCBA-- --===============0353770772== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 --===============0353770772==-- From nfsv4-bounces@ietf.org Tue Sep 04 21:06:12 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISjJe-0002Y9-O6; Tue, 04 Sep 2007 21:04:23 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IQXRs-0007yM-UZ for nfsv4@ietf.org; Wed, 29 Aug 2007 19:59:51 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IQXRo-0006YS-EH for nfsv4@ietf.org; Wed, 29 Aug 2007 19:59:48 -0400 X-IronPort-AV: E=Sophos;i="4.19,323,1183359600"; d="scan'208,217";a="98369327" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 29 Aug 2007 16:59:42 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l7TNxft6024426 for ; Wed, 29 Aug 2007 16:59:41 -0700 (PDT) Received: from exsvlrb02.hq.netapp.com ([10.56.8.63]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 29 Aug 2007 16:59:41 -0700 Received: from exsvl02.hq.netapp.com ([10.56.8.60]) by exsvlrb02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 29 Aug 2007 16:59:41 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 29 Aug 2007 16:59:41 -0700 Message-ID: <044B81DE141D7443BCE91E8F44B3C1E206D8C987@exsvl02.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Review comments for Locking and Directory DelegationsReview # 2 Thread-Index: AcfqmKoCUfhDSgluQG6IsAoSHMwoLw== From: "Khan, Saadia" To: X-OriginalArrivalTime: 29 Aug 2007 23:59:41.0831 (UTC) FILETIME=[AA696170:01C7EA98] X-Spam-Score: -4.0 (----) X-Scan-Signature: ccb50a2300954d38f2cc635c70f64dce X-Mailman-Approved-At: Tue, 04 Sep 2007 21:04:17 -0400 Subject: [nfsv4] Review comments for Locking and Directory DelegationsReview # 2 X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1997227766==" Errors-To: nfsv4-bounces@ietf.org This is a multi-part message in MIME format. --===============1997227766== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C7EA98.AA445159" This is a multi-part message in MIME format. ------_=_NextPart_001_01C7EA98.AA445159 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The second review meeting for locking and directory delegations was held on August 27, 2007. =20 The roles in this meeting were: - moderator - Shepler - reader - Eisler - scribe - Khan - author - Noveck - inspectors: Erasani, Gordon, Kustarz In this meeting, we covered lines 8751 - 9446, 9530 - 9728, 10476 - 10519 from draft 13. I am listing the review comments received during the review by name, line number and comment. =20 lines 8751 - 9446 =20 Khan - L 8775 - It should be emphasized that the server must not allow reclaims to happen if it is not storing the lock information on stable storage in order to be spec compliant. =20 Kustarz - L 8781 - Clarify what 'false positives' means here. =20 Noveck - L 8826 - Another case missing here is the one for a persistent session where the state is lost and the client is asked to reclaim its state. =20 Kustarz - General Comment - All state related errors should be consolidated and explained together, maybe in a separate section. =20 Eisler - L 8834 - s/as discussed above/as discussed in section blah. =20 Eisler - L 8838 - These lines are not correct and need to clarify what some of the SEQUENCE flags do. =20 Noveck - General comment - The two edge conditions mentioned in 8.4.3 are only in terms of lease expiration and not in the context of server revocation of locks. That needs to be addressed either in this section or this section needs to be somehow part of 8.4.3. =20 Ersani/Eisler - General comment - In the desription of test_stateid, it should be mentioned that the client may do this op whenever it wants, but it should definitely do this when any locks have been revoked and the server should allow the client to issue this op multiple times. =20 Eisler - L 8899, 8917 - s/lock/lease =20 Eisler - L 8978 - server should ignore these fields and not return an error. =20 Noveck - L 8971 - Mention here that the cleintid in CREATE_SESSION and DESTROY_SESSION is not ignored. =20 Eisler - L 8984 - OPEN and CREATE should be in lower case. =20 Eisler - L 8994 - s/lock/byte range lock =20 Khan/Kustarz - L 9004 - byte range locks should either be referred to as byte range locks or record locks in all places, currently this is not consistent. Same for octet locks. =20 Noveck - L 9029 - This should be mentioned in the previous chapter. =20 Kustarz - General Comment - Clarify all references to owners to be specifically lock owner/open owner or state owner. =20 Eisler - L 9051 - special stateid section should be referenced here. =20 Noveck - L 9132 - s/OPEN/stateid =20 Gordon - Comment for this section - replace setattr for set size with write. =20 Eisler - L 9176 - remove the line. =20 Eisler - L 9157 - There is no difference between using special stateids or regular stateids for this scenario. =20 Eisler - L 9254 - This is not true, it should be clarified what the new behavior is. =20 Eisler - L 9318 - Needs clarification. =20 Eisler - L 9381 - Section on stateid creation should be referenced and explained what happens to the old stateid in this case. =20 Eisler - L 9403 - s/open/OPENs =20 Kustarz - L 9403 - reference to the section that discusses how parallel opens should be handled and implemented. =20 Eisler - L 9413 - s/ay/may =20 Eisler - L 9417 - inadvertently (typo) =20 Noveck - L 9435 - This needs to be discussed on the mailing list. There could be issues here with part of the request being processed by one server instance and part of it by another one. =20 lines 9530 - 9728 =20 Eisler - L 9554 - s/callback/back channel =20 Khan - L 9581 - Clarify that nfs4err_delay could be returned in this case as well. =20 Eisler - L 9620 - s/leases/lease =20 Khan - L 9651 - s/but/and =20 Eisler - L 9693 - s/reclaim a/reclaim via a =20 Eisler - L 9701 - remove quotes. =20 Gordon - L 9648 - CLAIM_DELEGATE_PREV_FH should be mentioned here as well. =20 Eisler - L 9705 - s/freeing/revocation =20 Eisler - L 9723 - Needs to be clarified if the state needs to be in stable storage, the proper section should be referenced here. =20 lines 10476 - 10519=20 =20 Noveck - L 10510 - Needs to be clarified. =20 Khan - L 10516 - s/leases/lease =20 Eisler - General comment - It should be mentioned somewhere that the client needs to look at the status flags returned from SEQUENCE because it might need to do some actions based on what it got. =20 Gordon - L 10504 - s/a reasonable time/atleast one lease period =20 Khan - L 10534 - It should be added here that any open/lock state subsumed by the delegation does not get revoked when the delegation gets revoked. =20 Kustarz - General comment - It should be mentioned somewhere that when doing IO the order of checking stateids should be delegation/lock/open/special. =20 =20 thanks. Saadia =20 =20 ------_=_NextPart_001_01C7EA98.AA445159 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

The = second review=20 meeting for locking and directory delegations was held on August 27,=20 2007.

The roles in=20 this meeting were:

- moderator - Shepler

- = reader -=20 Eisler

- scribe - Khan

- author - Noveck

- inspectors: Erasani, Gordon, = Kustarz

In this=20 meeting, we covered lines 8751 -=20 9446, 9530 - 9728, 10476 - 10519 = from=20 draft 13.

I am listing=20 the review comments received during the review by name, line number and=20 comment.

lines 8751=20 - = 9446

Khan - L 8775 - It should be emphasized that = the server=20 must not allow reclaims to happen if it is not storing the lock = information on=20 stable storage in order to be spec compliant.

Kustarz - L 8781 - Clarify what 'false positives' = means=20 here.

Noveck - L 8826 - Another case missing here is the = one for a=20 persistent session where the state is lost and the client is asked to = reclaim=20 its state.

Kustarz - General Comment - All state related = errors should be=20 consolidated and explained together, maybe in a separate=20 section.

Eisler - L 8834 - s/as discussed above/as = discussed in section=20 blah.

Eisler - L 8838 - These lines are not correct and = need to=20 clarify what some of the SEQUENCE flags do.

Eisler - L 8899, 8917 -=20 s/lock/lease

Eisler - L 8978 - server should ignore these fields and = not return=20 an error.

Noveck - L 8971 - Mention here that the cleintid in = CREATE_SESSION and=20 DESTROY_SESSION is not=20 ignored.

Eisler - L 8984 - OPEN and CREATE should be in lower=20 case.

&= nbsp;

Eisler - L 8994 - s/lock/byte range=20 lock

<= /SPAN>

Khan/Kustarz - L 9004 - byte range locks should either be = referred to as=20 byte range locks or record locks in all places, currently this is not=20 consistent. Same for=20 octet locks.=

<= /SPAN>

Noveck - L 9029 - This should be mentioned in the previous=20 chapter.

<= /SPAN>

Kustarz - General Comment - Clarify all references to owners to = be=20 specifically lock owner/open owner or state=20 owner.

<= /SPAN>

Eisler - L 9051 - special stateid section should be referenced=20 here.

<= /SPAN>

Noveck - L 9132 -=20 s/OPEN/stateid

<= /SPAN>

Gordon - Comment for this section - replace setattr for set = size with=20 write.

<= /SPAN>

Eisler - L 9176 - remove the=20 line.

<= /SPAN>

Eisler - L 9157 - There is no difference between using special = stateids=20 or regular stateids for this=20 scenario.<= /SPAN>

<= /SPAN>

Eisler - L 9254 - This is not true, it should be clarified what = the new=20 behavior=20 is.=

<= /SPAN>=

Eisler - L 9318 - Needs=20 clarification.

<= /SPAN>=

Eisler - L 9381 - Section on stateid creation should be = referenced and=20 explained what happens to the old stateid in this=20 case.<= /SPAN>=

<= /SPAN>=

Eisler - L 9403 -=20 s/open/OPENs<= /SPAN>=

<= /SPAN>= &n= bsp;

Kustarz - L 9403 - reference to the section that discusses how = parallel=20 opens should be handled and=20 implemented.<= /SPAN>=

<= /SPAN>

Eisler - L 9413 -=20 s/ay/may

<= /SPAN>

Eisler - L 9417 - inadvertently=20 (typo)=

<= /SPAN>

<= /SPAN>=

lines 9530 -=20 9728=

Eisler - L 9554 - s/callback/back=20 channel

<= /SPAN>=

Khan - L 9581 - Clarify that nfs4err_delay could be returned in = this case=20 as=20 well.<= /SPAN>

<= /SPAN>=

Eisler - L 9620 -=20 s/leases/lease

<= /SPAN>=

Khan - L 9651 -=20 s/but/and<= /SPAN>=

<= /SPAN>=

Eisler - L 9693 - s/reclaim a/reclaim via=20 a<= /DIV>

<= /SPAN>=

Eisler - L 9701 - remove=20 quotes.<= /SPAN>

<= /SPAN>= &n= bsp;

Gordon - L 9648 - CLAIM_DELEGATE_PREV_FH should be mentioned = here as=20 well.<= /SPAN>=

<= /SPAN>= &n= bsp;

Eisler - L 9705 -=20 s/freeing/revocation= <= /SPAN>=

<= /SPAN>= <= /SPAN>

Eisler - L 9723 - Needs to be clarified if the state = needs to be in=20 stable storage, the proper section should be referenced=20 here.<= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>=

lines 10476 - 10519=20 <= /SPAN>

Noveck - L 10510 - Needs = to be=20 clarified.=

Khan - L 10516 -=20 s/leases/lease

Gordon - L 10504 - s/a reasonable = time/atleast one=20 lease=20 period=

Khan - L 10534 - It should be added here that = any=20 open/lock state subsumed by the delegation does not get revoked when the = delegation gets=20 revoked.<= /SPAN>

Kustarz - General = comment - It=20 should be mentioned somewhere that when doing IO the order of checking = stateids=20 should be=20 delegation/lock/open/special.<= /SPAN>= <= /SPAN>

= <= /SPAN>=

thanks.= <= /SPAN>=

Saadia<= /SPAN>= <= /SPAN>=

= <= /SPAN>=

The third review meeting for locking and = directory=20 delegations was held on September = 4,=20 2007.

The=20 roles in this meeting were:

- moderator - Shepler

- reader=20 - Eisler

- scribe - Khan

- author - Noveck

- inspectors: Erasani, Gordon, = Kustarz

In this=20 meeting, we covered lines 10547 - 10560, 13332 - 13478, 24310 - 24588, = 25788 -=20 25994 and 26044 - 26159 from draft=20 13.

I am listing=20 the review comments received during the review by name, line number and=20 comment.

lines 10547 -=20 10560

Eisler - L 10555 - It should be mentioned = that this=20 operation is valid for all file types except = dirs.

Khan - L 10557 - lock state should be = mentioned here as=20 well.

Eisler - Also mention that = callbacks can=20 be done in case the delegation is not immediately=20 available.

&nbs= p;

Ersani - Clarified that this = op can be=20 issued at reclaim time as = well.

&nbs= p;

lines 13332 -=20 13478

Eisler - Delegation related = text should be=20 in the caching chapter.

&nbs= p;

Eisler - L 13412 = -=20 s/nfsv4/nfsv4.1

Eisler - L 13411 - Clarify by mentioning that = the only=20 way to avoid delegation recall is to make changes through a session = associated=20 with the=20 clientid.<= /SPAN>

Eisler - L 13416 - add that = this is true=20 only if notifications are requested by the=20 client.

Eisler - L 13426 = - add the=20 third case where multiple clients hold delegations and one client makes = the=20 changes.

Eisler - L 13452 - mention CB_RECALL=20 here

Eisler - L 13456 - Remove the line starting = with 'Also=20 if=20 .."=

Saadia - L 13463 - Remove this = paragraph.=

Noveck=20 -=20 L 13477 - yes, the state should be=20 removed.

lines 24310 -=20 24588

Noveck<= /FONT> -=20 L 24312 - s/test/free one

Eisler - L 24353 - It should = be=20 mentioned in the implementation section that the client should test = the=20 stateid before freeing = it.

&nbs= p;

Noveck -=20 L 24336 - Add device id mapping here if there is consensus on that=20 issue.

&nbs= p;

Khan - L 24341 - Divide this = up into=20 multiple sentences for = clarity.

&nbs= p;

Eisler - L 24371 = - remove=20 (cfh)

Eisler - L 24386 - change to be a bitmap=20 instead.

Noveck - L 24461 -=20 remove 'also'

Khan - L 24467 - delete the last=20 sentence

Eisler - L 24479 - remove the part about = cookie=20 verifier

Eisler - specific = args/response fields=20 should be referenced in this=20 section

Khan - L 24493 - Lines = starting from=20 here in this paragraph should be moved to the implementation section and = '..'=20 should be changed to=20 parent.

Gordon - The server should not = add bits to=20 the supported notifications bitmask that the client did not ask=20 for.

Khan - L 24504 - Also mention = that if the=20 delegation is available, the server might decide to give it to the = client=20 instead of doing a callback=20 later.=

Khan - L 24519 -=20 s/flags/bits<= /FONT>

Eisler - L 24520 -=20 s/direction/directory

Khan - L 24541 - change = 'appropriately' to=20 mention what should be set=20 here.<= /SPAN>

Eisler - L 24588 - reference the section on = security=20 negotiations=20 here.<= /SPAN>=

Khan - L 24588 - This should = be clarified=20 and it should not say anything about setting hte=20 cfh.<= /FONT>

lines 25788 -=20 25994

Gordon - L 25794 -=20 s/error_codes<>/status_codes<>

Eisler - L 25839 = - some=20 examples and reference to the sequence op needed=20 here.<= /SPAN>

= &nb= sp;

Eisler - L 25844 -=20 s/provides/returns<= /SPAN>

Eisler - L 25847 - s/cause=20 processing/cause compound=20 processing=

Dave/Mike/Saadia - L 25855 - = Add=20 old_stateid and stale_stateid as valid errors. Also add EInval for = special=20 stateids.<= /SPAN>=

Eilser/Kustarz - Reference the = stateid=20 generation section in the implementation section.=20 <= /FONT>

Eisler - L 25865 - remove = (client=20 ID)= <= /FONT>=

= <= /SPAN>=

Eisler - L 25943 - It should = also mention=20 here that the server might decide to cancel the want by issuing a=20 cb_wants_cancelled=20 operation.= <= /SPAN>= <= /SPAN>=

= <= /SPAN>= <= /SPAN>

Eisler - L 25955 - The flags = here should=20 be=20 indented.<= /SPAN>= <= /SPAN>= <= /SPAN>=

= <= /SPAN>= <= /SPAN>=

= <= /FONT>= <= /FONT>= <= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>= <= /FONT>= <= /FONT>= <= /FONT>=

<= /SPAN>= <= /FONT>= <= /FONT>= <= /SPAN>= <= /SPAN>=

<= /SPAN>= <= /SPAN>= <= /FONT>= <= /FONT>= <= /FONT>=

<= /SPAN>= <= /SPAN>= <= /SPAN>= <= /SPAN>= <= /FONT>=

<= /SPAN>= lines 26044 - 26159 =

Khan - L 26074 -=20 s/of/or

Eisler - L 26084 - s/fs/file=20 system

Khan - L 26072 - It should be mentioned that = the client=20 MUST issue a reclaim_complete when its done.

Eisler - L 26103 -=20 s/must/MUST

Eisler - L 26116 - the two known edge = conditions should=20 be mentioned here and x-referenced.

thanks.

Saadia

<= /SPAN>

Hi=20 all,

I had = a question=20 regarding create session and lease renewals. Let's say the following = sequence of=20 actions happens:

TIME = =20 RPC

0 &nb= sp; =20 EXCHANGE_ID &n= bsp; //=20 Establishes clientid at server

1 &nb= sp; =20 CREATE_SESSION = ; //=20 Confirms clientid, creates session

2 &nb= sp; =20 GETATTR for lease time // Get lease time; value = =3D=20 60

...

25 &n= bsp; =20 CREATE_SESSION = ; //=20 Create another session

Now, = given that the=20 client has created a new session (and thus shown "proof of life") does = it mean=20 that the client lease is renewed or is the client still expected to send = a=20 SEQUENCE op before the old lease expiry time (time 62) in order to = ensure that=20 the client id/state is valid?

Thanks,

Regards

Rahul

------_=_NextPart_001_01C7EF5C.3C558759-- --===============1727916181== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 --===============1727916181==-- From Kawamuramia@IT-CORP.RU Tue Sep 04 21:41:54 2007 Return-path: Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISjty-0003T4-UO for nfsv4-archive@lists.ietf.org; Tue, 04 Sep 2007 21:41:54 -0400 Received: from adsl-122-184.37-151.net24.it ([151.37.184.122] helo=adsl-67-176.38-151.net24.it) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISjtx-0002pF-2f for nfsv4-archive@lists.ietf.org; Tue, 04 Sep 2007 21:41:54 -0400 Received: from utentepc by IT-CORP.RU with ASMTP id 1CF24B4E for ; Wed, 5 Sep 2007 03:42:00 +0200 Received: from utentepc ([166.150.139.49]) by IT-CORP.RU with ESMTP id 390346EFDACD for ; Wed, 5 Sep 2007 03:42:00 +0200 Message-ID: <000501c7ef5d$ee1a0c70$43b02697@utentepc> From: "Eti Kawamura" To: Subject: negolais Date: Wed, 5 Sep 2007 03:41:50 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0007_01C7EF6E.B1A2DC70" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 3.7 (+++) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 ------=_NextPart_000_0007_01C7EF6E.B1A2DC70 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.dggslaw.com/ Wazzup nfsv4-archive My wife complains about my small cock ALL THE TIME! bentley Finn ------=_NextPart_000_0007_01C7EF6E.B1A2DC70 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.dggslaw.com/

Wazzup nfsv4-archive

My wife complains about my small cock ALL THE = TIME!

bentley Finn

------=_NextPart_000_0007_01C7EF6E.B1A2DC70-- From nfsv4-bounces@ietf.org Wed Sep 05 07:23:24 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISswy-00020f-EA; Wed, 05 Sep 2007 07:21:36 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISsww-00020X-JE for nfsv4@ietf.org; Wed, 05 Sep 2007 07:21:34 -0400 Received: from gw-colo-pa.panasas.com ([66.238.117.130] helo=cassoulet.panasas.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ISsww-0001sR-8l for nfsv4@ietf.org; Wed, 05 Sep 2007 07:21:34 -0400 Received: from riverside.int.panasas.com (localhost.localdomain [127.0.0.1]) by cassoulet.panasas.com (8.13.1/8.13.1) with ESMTP id l85BLV3t006130; Wed, 5 Sep 2007 07:21:31 -0400 Received: from 172.17.1.41 ([172.17.1.41] helo=riverside.int.panasas.com) by ASSP-nospam; 5 Sep 2007 07:21:31 -0400 Received: from bh-buildlin1.bhalevy.com ([172.17.28.104]) by riverside.int.panasas.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 07:21:31 -0400 Message-ID: <46DE9139.20002@panasas.com> Date: Wed, 05 Sep 2007 14:21:29 +0300 From: Benny Halevy User-Agent: Thunderbird 2.0.0.6 (X11/20070728) MIME-Version: 1.0 To: "Iyer, Rahul" Subject: Re: [nfsv4] Create session and lease renewal References: <7619F3097FAB384287CF636FF92D0CA10ABC2608@exsvl01.hq.netapp.com> In-Reply-To: <7619F3097FAB384287CF636FF92D0CA10ABC2608@exsvl01.hq.netapp.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Sep 2007 11:21:31.0862 (UTC) FILETIME=[E92B3760:01C7EFAE] X-Spam-Score: 0.0 (/) X-Scan-Signature: a7d6aff76b15f3f56fcb94490e1052e4 Cc: nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org Although it makes sense to renew the lease on CREATE_SESSION I think it's simpler if it does not do that by itself. The client can always precede CREATE_SESSION with SEQUENCE if it intends to trunk it to the same client ID. Benny On Sep 05, 2007, 4:29 +0300, "Iyer, Rahul" wrote: > Hi all, > I had a question regarding create session and lease renewals. Let's say > the following sequence of actions happens: > > TIME RPC > 0 EXCHANGE_ID // Establishes > clientid at server > 1 CREATE_SESSION // Confirms clientid, > creates session > 2 GETATTR for lease time // Get lease time; > value = 60 > ... > 25 CREATE_SESSION // Create another session > > > Now, given that the client has created a new session (and thus shown > "proof of life") does it mean that the client lease is renewed or is the > client still expected to send a SEQUENCE op before the old lease expiry > time (time 62) in order to ensure that the client id/state is valid? > > Thanks, > Regards > Rahul > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 08:15:27 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IStlM-00020W-BR; Wed, 05 Sep 2007 08:13:40 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IStlL-0001yH-1Y for nfsv4@ietf.org; Wed, 05 Sep 2007 08:13:39 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IStlJ-0001Ie-Oa for nfsv4@ietf.org; Wed, 05 Sep 2007 08:13:39 -0400 X-IronPort-AV: E=Sophos;i="4.20,210,1186383600"; d="scan'208";a="100394374" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 05 Sep 2007 05:13:37 -0700 Received: from svlexrs01.hq.netapp.com (svlexrs01.corp.netapp.com [10.57.156.158]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l85CDae2000956; Wed, 5 Sep 2007 05:13:36 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 05:13:36 -0700 Received: from exnane01.hq.netapp.com ([10.97.0.61]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 05:13:36 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [nfsv4] Create session and lease renewal Date: Wed, 5 Sep 2007 08:13:34 -0400 Message-ID: In-Reply-To: <46DE9139.20002@panasas.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nfsv4] Create session and lease renewal Thread-Index: Acfvrzi6bbxs6lBxSTGPMq02GLHF6AABWr7w From: "Noveck, Dave" To: "Benny Halevy" , "Iyer, Rahul" X-OriginalArrivalTime: 05 Sep 2007 12:13:36.0368 (UTC) FILETIME=[2F84FF00:01C7EFB6] X-Spam-Score: -4.0 (----) X-Scan-Signature: 10d3e4e3c32e363f129e380e644649be Cc: nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org We discussed a related issue at the locking review. The thing about preceeding this with a sequence is that strictly speaking, the COMPOUND is not atomic so the time of the sequence and the CREATE_SESSION (for example) might not be the same. So I have been thinking that the rule should be that you renew the lease: At SEQUENCE. At any operation which creates a piece of locking state. Otherwise, we might have the odd situation that we create a lock which is expired from the moment is created, and that just seems weird. The reason that this hooks up with your question, is that we have also been considering the question of whether having a session means you have lease. In other words, is a session to be considered as state. In v4.0, the model was that you didn't have a lease until you had some type of lock. But now, given that SEQUENCE renews the lease, it's kind of odd to have it renew the lease when you might not have one. So given all that, I'm thinking that as part of the update for the locking review, we say the lease is renewed at SEQUENCE and at any operation that creates state and that having a session means you do have a piece of state. So the answer to whether CREATE_SESSION renews/creates a lease would be "Yes". So if there is no objection, that's what I'm going to do.=20 -----Original Message----- From: Benny Halevy [mailto:bhalevy@panasas.com]=20 Sent: Wednesday, September 05, 2007 7:21 AM To: Iyer, Rahul Cc: nfsv4@ietf.org Subject: Re: [nfsv4] Create session and lease renewal Although it makes sense to renew the lease on CREATE_SESSION I think it's simpler if it does not do that by itself. The client can always precede CREATE_SESSION with SEQUENCE if it intends to trunk it to the same client ID. Benny=20 On Sep 05, 2007, 4:29 +0300, "Iyer, Rahul" wrote: > Hi all, > I had a question regarding create session and lease renewals. Let's=20 > say the following sequence of actions happens: > =20 > TIME RPC > 0 EXCHANGE_ID // Establishes > clientid at server > 1 CREATE_SESSION // Confirms clientid, > creates session > 2 GETATTR for lease time // Get lease time; > value =3D 60 > ... > 25 CREATE_SESSION // Create another session > =20 > =20 > Now, given that the client has created a new session (and thus shown=20 > "proof of life") does it mean that the client lease is renewed or is=20 > the client still expected to send a SEQUENCE op before the old lease=20 > expiry time (time 62) in order to ensure that the client id/state is valid? > =20 > Thanks, > Regards > Rahul > =20 >=20 >=20 >=20 > ---------------------------------------------------------------------- > -- >=20 > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From Saman-duffin@7himmel.dk Wed Sep 05 08:33:19 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISu4N-0000OR-J0 for nfsv4-archive@lists.ietf.org; Wed, 05 Sep 2007 08:33:19 -0400 Received: from [60.54.32.129] (helo=[60.54.32.129]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ISu4L-0003wC-Qc for nfsv4-archive@lists.ietf.org; Wed, 05 Sep 2007 08:33:19 -0400 Received: by 10.178.205.18 with SMTP id NvQszbYujWhZX; Wed, 5 Sep 2007 20:33:11 +0800 (GMT) Received: by 192.168.173.29 with SMTP id pgVWDDLfALChZk.0670055644432; Wed, 5 Sep 2007 20:33:09 +0800 (GMT) Message-ID: <000901c7efb8$e8ecea50$8120363c@user> From: "Saman duffin" To: Subject: ogawkcub Date: Wed, 5 Sep 2007 20:33:06 +0800 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0007_01C7EFFB.F7102A50" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 0.1 (/) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 ------=_NextPart_000_0007_01C7EFFB.F7102A50 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.devguur.com/ Yo duffin Are you really satisfied with your manhood size? EDRIC Heering ------=_NextPart_000_0007_01C7EFFB.F7102A50 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.devguur.com/

Yo duffin

Are you really satisfied with your manhood = size?

EDRIC Heering

------=_NextPart_000_0007_01C7EFFB.F7102A50-- From nfsv4-bounces@ietf.org Wed Sep 05 08:47:02 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISuFw-0001vM-Jp; Wed, 05 Sep 2007 08:45:16 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISuFv-0001vF-8B for nfsv4@ietf.org; Wed, 05 Sep 2007 08:45:15 -0400 Received: from gw-colo-pa.panasas.com ([66.238.117.130] helo=cassoulet.panasas.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ISuFu-0004JZ-Nk for nfsv4@ietf.org; Wed, 05 Sep 2007 08:45:15 -0400 Received: from riverside.int.panasas.com (localhost.localdomain [127.0.0.1]) by cassoulet.panasas.com (8.13.1/8.13.1) with ESMTP id l85CjEBZ007953; Wed, 5 Sep 2007 08:45:14 -0400 Received: from 172.17.1.41 ([172.17.1.41] helo=riverside.int.panasas.com) by ASSP-nospam; 5 Sep 2007 08:45:14 -0400 Received: from bh-buildlin1.bhalevy.com ([172.17.28.104]) by riverside.int.panasas.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 08:45:13 -0400 Message-ID: <46DEA4D8.6010707@panasas.com> Date: Wed, 05 Sep 2007 15:45:12 +0300 From: Benny Halevy User-Agent: Thunderbird 2.0.0.6 (X11/20070728) MIME-Version: 1.0 To: "Noveck, Dave" Subject: Re: [nfsv4] Create session and lease renewal References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Sep 2007 12:45:14.0196 (UTC) FILETIME=[9AB69140:01C7EFBA] X-Spam-Score: 0.0 (/) X-Scan-Signature: 14582b0692e7f70ce7111d04db3781c8 Cc: "Iyer, Rahul" , nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org I have no objection. If you say the sessionid represents state the same as other types of stateids it even makes sense :-) Benny On Sep 05, 2007, 15:13 +0300, "Noveck, Dave" wrote: > We discussed a related issue at the locking review. The thing about > preceeding this with a sequence is that strictly speaking, the COMPOUND > is not atomic so the time of the sequence and the CREATE_SESSION (for > example) might not be the same. > > So I have been thinking that the rule should be that you renew the > lease: > > At SEQUENCE. > > At any operation which creates a piece of locking state. > > Otherwise, we might have the odd situation that we create a lock which > is expired from the moment is created, and that just seems weird. > > The reason that this hooks up with your question, is that we have also > been considering the question of whether having a session means you have > lease. In other words, is a session to be considered as state. In > v4.0, the model was that you didn't have a lease until you had some type > of lock. But now, given that SEQUENCE renews the lease, it's kind of > odd to have it renew the lease when you might not have one. > > So given all that, I'm thinking that as part of the update for the > locking review, we say the lease is renewed at SEQUENCE and at any > operation that creates state and that having a session means you do have > a piece of state. So the answer to whether CREATE_SESSION > renews/creates a lease would be "Yes". > > So if there is no objection, that's what I'm going to do. > > -----Original Message----- > From: Benny Halevy [mailto:bhalevy@panasas.com] > Sent: Wednesday, September 05, 2007 7:21 AM > To: Iyer, Rahul > Cc: nfsv4@ietf.org > Subject: Re: [nfsv4] Create session and lease renewal > > Although it makes sense to renew the lease on CREATE_SESSION I think > it's simpler if it does not do that by itself. > The client can always precede CREATE_SESSION with SEQUENCE if it intends > to trunk it to the same client ID. > > Benny > > On Sep 05, 2007, 4:29 +0300, "Iyer, Rahul" > wrote: >> Hi all, >> I had a question regarding create session and lease renewals. Let's >> say the following sequence of actions happens: >> >> TIME RPC >> 0 EXCHANGE_ID // Establishes >> clientid at server >> 1 CREATE_SESSION // Confirms clientid, >> creates session >> 2 GETATTR for lease time // Get lease time; >> value = 60 >> ... >> 25 CREATE_SESSION // Create another > session >> >> >> Now, given that the client has created a new session (and thus shown >> "proof of life") does it mean that the client lease is renewed or is >> the client still expected to send a SEQUENCE op before the old lease >> expiry time (time 62) in order to ensure that the client id/state is > valid? >> >> Thanks, >> Regards >> Rahul >> >> >> >> >> ---------------------------------------------------------------------- >> -- >> >> _______________________________________________ >> nfsv4 mailing list >> nfsv4@ietf.org >> https://www1.ietf.org/mailman/listinfo/nfsv4 > > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From Cardyvve@08470.com Wed Sep 05 08:57:19 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISuRb-0003TJ-IN for nfsv4-archive@lists.ietf.org; Wed, 05 Sep 2007 08:57:19 -0400 Received: from 183.99.48.60.wmu01-home.tm.net.my ([60.48.99.183] helo=[60.54.32.129]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ISuRZ-0004iU-LL for nfsv4-archive@lists.ietf.org; Wed, 05 Sep 2007 08:57:19 -0400 Received: from user by 08470.com with ASMTP id CBB34AD9 for ; Wed, 5 Sep 2007 20:57:36 +0800 Received: from user ([100.181.186.55]) by 08470.com with ESMTP id 074BDF2A0013 for ; Wed, 5 Sep 2007 20:57:36 +0800 Message-ID: <000b01c7efbc$44af7cb0$8120363c@user> From: "Franny Cardy" To: Subject: offerti Date: Wed, 5 Sep 2007 20:57:08 +0800 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0004_01C7EFFF.52D2BCB0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 1.7 (+) X-Scan-Signature: 97adf591118a232206bdb5a27b217034 ------=_NextPart_000_0004_01C7EFFF.52D2BCB0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.derheini.com/ How it is going Cardy VERY easy and VERY effective cock nlargement Cosma Crampton ------=_NextPart_000_0004_01C7EFFF.52D2BCB0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.derheini.com/

How it is going Cardy

VERY easy and VERY effective cock = nlargement

Cosma Crampton

------=_NextPart_000_0004_01C7EFFF.52D2BCB0-- From nfsv4-bounces@ietf.org Wed Sep 05 09:21:16 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISun5-0004AJ-93; Wed, 05 Sep 2007 09:19:31 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISun3-000458-AS for nfsv4@ietf.org; Wed, 05 Sep 2007 09:19:29 -0400 Received: from web31807.mail.mud.yahoo.com ([68.142.207.70]) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1ISun2-0005QU-Ur for nfsv4@ietf.org; Wed, 05 Sep 2007 09:19:29 -0400 Received: (qmail 38994 invoked by uid 60001); 5 Sep 2007 13:19:26 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=ggJYVul+nb9SS3/S9ecdTJiKwJMVIVdXz8o8JPM3HamNVILbIoibUFJVvN7e1P2+D9HcdK7AqvHvuaem8tr4DAILDFhsP794ICpHfgBLrXo9Sgvm1ODereyrJtT3X2DwI0v4kKIG5ZCy8hCazIzzvTPoK8yb+b7Q6cS7bekkKGY=; X-YMail-OSG: ysvNZtAVM1kd9e.pUo7RZZ3_ThoThSF2ehdpAP3Zgo7NJGgg2E0bUSDcaRyEvfO2toZ394mXRQ-- Received: from [198.95.226.224] by web31807.mail.mud.yahoo.com via HTTP; Wed, 05 Sep 2007 06:19:26 PDT Date: Wed, 5 Sep 2007 06:19:26 -0700 (PDT) From: Mike Eisler Subject: RE: [nfsv4] Create session and lease renewal To: nfsv4@ietf.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <314536.38976.qm@web31807.mail.mud.yahoo.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: ea4ac80f790299f943f0a53be7e1a21a X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: email2mre-ietf@yahoo.com List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org --- "Noveck, Dave" wrote: > So given all that, I'm thinking that as part of the update for the > locking review, we say the lease is renewed at SEQUENCE and at any > operation that creates state and that having a session means you do > have > a piece of state. So the answer to whether CREATE_SESSION > renews/creates a lease would be "Yes". > > So if there is no objection, that's what I'm going to do. Not just creating state. Say I have: SEQUENCE, PUTFH, READ In NFSv4.0 we'd have just had: PUTFH, READ and in NFSv4.0 the READ, because it took a stateid, renewed the lease. Wouldn't be simpler to state that the lease is renewed at SEQUENCE, and MUST NOT expire during the processing of the remaining operations in the COMPOUND, and MUST be renewed after the COMPOUND is executed. _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 09:38:06 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISv3N-00006x-4E; Wed, 05 Sep 2007 09:36:21 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISv3L-000057-3M for nfsv4@ietf.org; Wed, 05 Sep 2007 09:36:20 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ISv3K-0005vp-Kp for nfsv4@ietf.org; Wed, 05 Sep 2007 09:36:19 -0400 X-IronPort-AV: E=Sophos;i="4.20,210,1186383600"; d="scan'208";a="100413603" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 05 Sep 2007 06:36:02 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l85Da2mi024405; Wed, 5 Sep 2007 06:36:02 -0700 (PDT) Received: from exsvlrb02.hq.netapp.com ([10.56.8.63]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 06:36:02 -0700 Received: from exnane01.hq.netapp.com ([10.97.0.61]) by exsvlrb02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 06:36:02 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [nfsv4] Create session and lease renewal Date: Wed, 5 Sep 2007 09:36:00 -0400 Message-ID: In-Reply-To: <314536.38976.qm@web31807.mail.mud.yahoo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nfsv4] Create session and lease renewal Thread-Index: Acfvv6sb9BQT84i2Ry29lRkd8PaVvAAAG80Q From: "Noveck, Dave" To: , X-OriginalArrivalTime: 05 Sep 2007 13:36:02.0525 (UTC) FILETIME=[B3A8A8D0:01C7EFC1] X-Spam-Score: 0.0 (/) X-Scan-Signature: 92df29fa99cf13e554b84c8374345c17 Cc: X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org > Not just creating state. Say I have: > > SEQUENCE, PUTFH, READ > > In NFSv4.0 we'd have just had: > > PUTFH, READ > > and in NFSv4.0 the READ, because it took a stateid, renewed the lease. True but we are not under any obligation that I can see to renew the lease under all circumstances under which it would have been renewed in v4.0.=20 The case in which we create state is different in that it has the potential issue that you can create state with an expired lease (but which didn't exist when the lease was itself expired) so that the description of lease expiration and lease revival has one more really nasty case to deal with. > Wouldn't be simpler to state that the lease is renewed at=20 > SEQUENCE, and MUST NOT expire during the processing of the=20 > remaining operations in the COMPOUND, and MUST be renewed=20 > after the COMPOUND is executed.=20 It is simpler to say, but I'm not sure that the resulting protocol is simpler (or even as simple). So you could have a lease renewed (at SEQUENCE), the lease expire, have a lock created with a lease that is expired (so maybe it vanishes as soon it is created?) and then the lease is revived when the COMPOUND ends. I'd like not to have describe all the possibilities that that raises. I think it is simpler if creation of state is a lease-renewing/reviving event. -----Original Message----- From: Mike Eisler [mailto:email2mre-ietf@yahoo.com]=20 Sent: Wednesday, September 05, 2007 9:19 AM To: nfsv4@ietf.org Subject: RE: [nfsv4] Create session and lease renewal --- "Noveck, Dave" wrote: > So given all that, I'm thinking that as part of the update for the=20 > locking review, we say the lease is renewed at SEQUENCE and at any=20 > operation that creates state and that having a session means you do=20 > have a piece of state. So the answer to whether CREATE_SESSION=20 > renews/creates a lease would be "Yes". >=20 > So if there is no objection, that's what I'm going to do.=20 Not just creating state. Say I have: SEQUENCE, PUTFH, READ In NFSv4.0 we'd have just had: PUTFH, READ and in NFSv4.0 the READ, because it took a stateid, renewed the lease.=20 Wouldn't be simpler to state that the lease is renewed at SEQUENCE, and MUST NOT expire during the processing of the remaining operations in the COMPOUND, and MUST be renewed after the COMPOUND is executed. _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 10:51:19 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwCA-0007FH-I3; Wed, 05 Sep 2007 10:49:30 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwC9-0007FB-1K for nfsv4@ietf.org; Wed, 05 Sep 2007 10:49:29 -0400 Received: from web31812.mail.mud.yahoo.com ([68.142.207.75]) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1ISwC8-0007ss-DV for nfsv4@ietf.org; Wed, 05 Sep 2007 10:49:28 -0400 Received: (qmail 61396 invoked by uid 60001); 5 Sep 2007 14:49:27 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=YOQdx+4y3cKWTy4Na1nmn7cuHjL+MBegbEAEsZ6eYogxEOSIRFr+AClQ8QOi4vQk7Ngq84DXQBYVCxVS9PCWmJhNW3N07d/+5iZNikdl6DRhN0Cs0a0TUlHBg8pL2sLX+0pBrEh0dMTD87ISD7d4O+N9NiiB6q9nXWrH2FQgB7E=; X-YMail-OSG: SkhOHT4VM1lWpfd_Pg0zL2wyfsobNNWxERxiGICtl4nQqiC.N86MN3wQGjwG7uh9QeoniBFaHCjgiMWkR2NS3eLydwMnR_2siS.wUhjt5bqKqDwQdNc- Received: from [198.95.226.224] by web31812.mail.mud.yahoo.com via HTTP; Wed, 05 Sep 2007 07:49:27 PDT Date: Wed, 5 Sep 2007 07:49:27 -0700 (PDT) From: Mike Eisler Subject: RE: [nfsv4] Create session and lease renewal To: nfsv4@ietf.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <390555.61323.qm@web31812.mail.mud.yahoo.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: 825e642946eda55cd9bc654a36dab8c2 X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: email2mre-ietf@yahoo.com List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org --- "Noveck, Dave" wrote: > > Not just creating state. Say I have: > > > > SEQUENCE, PUTFH, READ > > > > In NFSv4.0 we'd have just had: > > > > PUTFH, READ > > > > and in NFSv4.0 the READ, because it took a stateid, renewed the > lease. > > True but we are not under any obligation that I can see to renew the > lease under all circumstances under which it would have been renewed > in > v4.0. So if the lease attribute is 60 seconds, at time T SEQUENCE is processed, and at time T+61 the READ is processed, I lose my lease, my locks, and the READ fails? It should be made clear in the spec if that's what we really want to design. I think that when this situation comes up, customers will be irate at the insult (lost lease) to injury (because a server takes 61 seconds to process a PUTFH ... it could happen if a file system is offline, or LDAP is offline during the export policy check) and server implementors will rapidly adopt my model. I know that realistically NFS4ERR_DELAY will be returned if PUTFH really takes 61 seconds, but doesn't detract much my point. If the server aborts PUTFH after say 20 seconds, then unless COMPOUND renews the lease at the end of processing, the client has just 40 seconds of lease left. So because the server, which is already slow, fails to renew the lease, the client has to send SEQUENCE more frequently, adding more load to the server, and exacerbating the situation. > > Wouldn't be simpler to state that the lease is renewed at > > SEQUENCE, and MUST NOT expire during the processing of the > > remaining operations in the COMPOUND, and MUST be renewed > > after the COMPOUND is executed. > > It is simpler to say, but I'm not sure that the resulting protocol is > simpler (or even as simple). So you could have a lease renewed (at > SEQUENCE), the lease expire, have a lock created with a lease that is I wrote: "MUST NOT expire during processing of the remaining COMPOUND". If the lease is renewed at SEQUENCE, under my suggestion it expires no sooner than the wall clock time at when the COMPOUND completes plus the lease attribute. We already require the server to associate the sessionid (and the implied client ID) of the SEQUENCE with all remaining operations in the COMPOUND, and we already require that a COMPOUND have just one SEQUENCE. Further requring the timer that is the lease to be reset to the value of the lease attribute, when SEQUENCE is seen, then paused(i.e. not flow backwards) during the COMPOUND processing and then resuming the timer when the processing stops is really a small thing by comparison. > expired (so maybe it vanishes as soon it is created?) and then the > lease > is revived when the COMPOUND ends. I'd like not to have describe all > the possibilities that that raises. I think it is simpler if > creation > of state is a lease-renewing/reviving event. > > -----Original Message----- > From: Mike Eisler [mailto:email2mre-ietf@yahoo.com] > Sent: Wednesday, September 05, 2007 9:19 AM > To: nfsv4@ietf.org > Subject: RE: [nfsv4] Create session and lease renewal > > > --- "Noveck, Dave" wrote: > > > > So given all that, I'm thinking that as part of the update for the > > locking review, we say the lease is renewed at SEQUENCE and at any > > operation that creates state and that having a session means you do > > > have a piece of state. So the answer to whether CREATE_SESSION > > renews/creates a lease would be "Yes". > > > > So if there is no objection, that's what I'm going to do. > > Not just creating state. Say I have: > > SEQUENCE, PUTFH, READ > > In NFSv4.0 we'd have just had: > > PUTFH, READ > > and in NFSv4.0 the READ, because it took a stateid, renewed the > lease. > > Wouldn't be simpler to state that the lease is renewed at SEQUENCE, > and > MUST NOT expire during the processing of the remaining operations in > the > COMPOUND, and MUST be renewed after the COMPOUND is executed. > > > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 > _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:06:39 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwR3-0001Sg-Fz; Wed, 05 Sep 2007 11:04:53 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwR2-0001SV-V8 for nfsv4@ietf.org; Wed, 05 Sep 2007 11:04:52 -0400 Received: from brmea-mail-1.sun.com ([192.18.98.31]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISwR1-0005pU-KH for nfsv4@ietf.org; Wed, 05 Sep 2007 11:04:52 -0400 Received: from fe-amer-05.sun.com ([192.18.108.179]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l85F4oc5024729 for ; Wed, 5 Sep 2007 15:04:50 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNW00501HLMP300@mail-amer.sun.com> (original mail from Spencer.Shepler@Sun.COM) for nfsv4@ietf.org; Wed, 05 Sep 2007 09:04:50 -0600 (MDT) Received: from [192.168.0.5] (adsl-71-145-147-60.dsl.austtx.sbcglobal.net [71.145.147.60]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNW00JZIHW1SK00@mail-amer.sun.com>; Wed, 05 Sep 2007 09:04:50 -0600 (MDT) Date: Wed, 05 Sep 2007 10:05:12 -0500 From: Spencer Shepler Subject: Re: [nfsv4] Create session and lease renewal In-reply-to: <390555.61323.qm@web31812.mail.mud.yahoo.com> To: email2mre-ietf@yahoo.com Message-id: <969BB655-0D7E-40AE-8634-1AEEFE055224@sun.com> MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; format=flowed; delsp=yes; charset=US-ASCII Content-transfer-encoding: 7BIT References: <390555.61323.qm@web31812.mail.mud.yahoo.com> X-Spam-Score: -1.0 (-) X-Scan-Signature: 6640e3bbe8a4d70c4469bcdcbbf0921d Cc: nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org I agree with this assessment, Mike. It is easier to understand even in the environment that COMPOUND is not atomic in nature. The COMPOUND is process as a whole and for very odd, edge case reasons, it may take a very long time (longer than the lease). Spencer On Sep 5, 2007, at 9:49 AM, Mike Eisler wrote: > > --- "Noveck, Dave" wrote: > >>> Not just creating state. Say I have: >>> >>> SEQUENCE, PUTFH, READ >>> >>> In NFSv4.0 we'd have just had: >>> >>> PUTFH, READ >>> >>> and in NFSv4.0 the READ, because it took a stateid, renewed the >> lease. >> >> True but we are not under any obligation that I can see to renew the >> lease under all circumstances under which it would have been renewed >> in >> v4.0. > > So if the lease attribute is 60 seconds, at time T SEQUENCE is > processed, and at time T+61 the READ is processed, I lose my lease, > my locks, and the READ fails? > > It should be made clear in the spec if that's what we really want > to design. > > I think that when this situation comes up, customers will be irate > at the insult (lost lease) to injury (because a server takes 61 > seconds to process a PUTFH ... it could happen if a file system is > offline, or LDAP is offline during the export policy check) and server > implementors will rapidly > adopt my model. > > I know that realistically NFS4ERR_DELAY will be returned if PUTFH > really takes 61 seconds, but doesn't detract much my point. If the > server aborts PUTFH after say 20 seconds, then unless COMPOUND > renews the lease at the end of processing, the client has just > 40 seconds of lease left. So because the server, which is already > slow, fails to renew the lease, the client has to send SEQUENCE > more frequently, adding more load to the server, and exacerbating > the situation. > >>> Wouldn't be simpler to state that the lease is renewed at >>> SEQUENCE, and MUST NOT expire during the processing of the >>> remaining operations in the COMPOUND, and MUST be renewed >>> after the COMPOUND is executed. >> >> It is simpler to say, but I'm not sure that the resulting protocol is >> simpler (or even as simple). So you could have a lease renewed (at >> SEQUENCE), the lease expire, have a lock created with a lease that is > > I wrote: "MUST NOT expire during processing of the remaining > COMPOUND". If the lease is renewed at SEQUENCE, under my > suggestion it expires no sooner than the wall clock > time at when the COMPOUND > completes plus the lease attribute. > > We already require the server to associate the sessionid (and > the implied client ID) of the SEQUENCE > with all remaining operations in the COMPOUND, and we already require > that a COMPOUND have just one SEQUENCE. Further requring the timer > that is the lease to be reset to the value of the lease attribute, > when > SEQUENCE is seen, then paused(i.e. not flow backwards) during > the COMPOUND processing and then resuming the timer when the > processing > stops is really a small thing by comparison. > >> expired (so maybe it vanishes as soon it is created?) and then the >> lease >> is revived when the COMPOUND ends. I'd like not to have describe all >> the possibilities that that raises. I think it is simpler if >> creation >> of state is a lease-renewing/reviving event. >> >> -----Original Message----- >> From: Mike Eisler [mailto:email2mre-ietf@yahoo.com] >> Sent: Wednesday, September 05, 2007 9:19 AM >> To: nfsv4@ietf.org >> Subject: RE: [nfsv4] Create session and lease renewal >> >> >> --- "Noveck, Dave" wrote: >> >> >>> So given all that, I'm thinking that as part of the update for the >>> locking review, we say the lease is renewed at SEQUENCE and at any >>> operation that creates state and that having a session means you do >> >>> have a piece of state. So the answer to whether CREATE_SESSION >>> renews/creates a lease would be "Yes". >>> >>> So if there is no objection, that's what I'm going to do. >> >> Not just creating state. Say I have: >> >> SEQUENCE, PUTFH, READ >> >> In NFSv4.0 we'd have just had: >> >> PUTFH, READ >> >> and in NFSv4.0 the READ, because it took a stateid, renewed the >> lease. >> >> Wouldn't be simpler to state that the lease is renewed at SEQUENCE, >> and >> MUST NOT expire during the processing of the remaining operations in >> the >> COMPOUND, and MUST be renewed after the COMPOUND is executed. >> >> >> >> _______________________________________________ >> nfsv4 mailing list >> nfsv4@ietf.org >> https://www1.ietf.org/mailman/listinfo/nfsv4 >> > > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:08:02 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwSP-0001sM-V2; Wed, 05 Sep 2007 11:06:17 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwSO-0001sH-NI for nfsv4@ietf.org; Wed, 05 Sep 2007 11:06:16 -0400 Received: from brmea-mail-1.sun.com ([192.18.98.31]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISwSN-0005qz-FF for nfsv4@ietf.org; Wed, 05 Sep 2007 11:06:16 -0400 Received: from fe-amer-01.sun.com ([192.18.108.175]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l85F6FjR025596 for ; Wed, 5 Sep 2007 15:06:15 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNW00E01HLEOV00@mail-amer.sun.com> (original mail from Spencer.Shepler@Sun.COM) for nfsv4@ietf.org; Wed, 05 Sep 2007 09:06:15 -0600 (MDT) Received: from [192.168.0.5] (adsl-71-145-147-60.dsl.austtx.sbcglobal.net [71.145.147.60]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNW00CZOHYEEX20@mail-amer.sun.com>; Wed, 05 Sep 2007 09:06:15 -0600 (MDT) Date: Wed, 05 Sep 2007 10:06:37 -0500 From: Spencer Shepler Subject: Re: [nfsv4] Create session and lease renewal In-reply-to: To: "Noveck, Dave" Message-id: <898D147B-0AA5-44DE-B419-47F865EE4B8A@sun.com> MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; format=flowed; delsp=yes; charset=US-ASCII Content-transfer-encoding: 7BIT References: X-Spam-Score: -1.0 (-) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Cc: email2mre-ietf@yahoo.com, nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org On Sep 5, 2007, at 8:36 AM, Noveck, Dave wrote: >> Not just creating state. Say I have: >> >> SEQUENCE, PUTFH, READ >> >> In NFSv4.0 we'd have just had: >> >> PUTFH, READ >> >> and in NFSv4.0 the READ, because it took a stateid, renewed the >> lease. > > True but we are not under any obligation that I can see to renew the > lease under all circumstances under which it would have been > renewed in > v4.0. > > The case in which we create state is different in that it has the > potential issue that you can create state with an expired lease (but > which didn't exist when the lease was itself expired) so that the > description of lease expiration and lease revival has one more really > nasty case to deal with. Would you mind providing a scenario where this could happen so that we capture it in the archives so it is clear such that it doesn't end up as a piece of NFSv4.1 esoteric knowledge. Spencer _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:36:39 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwu5-0005kO-LH; Wed, 05 Sep 2007 11:34:53 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwu4-0005jS-LP for nfsv4@ietf.org; Wed, 05 Sep 2007 11:34:52 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISwu3-0006R5-87 for nfsv4@ietf.org; Wed, 05 Sep 2007 11:34:52 -0400 X-IronPort-AV: E=Sophos;i="4.20,210,1186383600"; d="scan'208";a="100452637" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 05 Sep 2007 08:34:50 -0700 Received: from svlexrs01.hq.netapp.com (svlexrs01.corp.netapp.com [10.57.156.158]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l85FYnQY007397; Wed, 5 Sep 2007 08:34:50 -0700 (PDT) Received: from exsvlrb02.hq.netapp.com ([10.56.8.63]) by svlexrs01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 08:34:49 -0700 Received: from exnane01.hq.netapp.com ([10.97.0.61]) by exsvlrb02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 08:34:49 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [nfsv4] Create session and lease renewal Date: Wed, 5 Sep 2007 11:34:48 -0400 Message-ID: In-Reply-To: <390555.61323.qm@web31812.mail.mud.yahoo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nfsv4] Create session and lease renewal Thread-Index: AcfvzGT/yzMmItMyQ1OYnkYv734GKQABHGtw From: "Noveck, Dave" To: , X-OriginalArrivalTime: 05 Sep 2007 15:34:49.0319 (UTC) FILETIME=[4B8F1F70:01C7EFD2] X-Spam-Score: -4.0 (----) X-Scan-Signature: d16ce744298aacf98517bc7c108bd198 Cc: X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org So here is a model that I think addresses my issue, the possiblity of creating a lock while a lease is expired, and your issues, the possiblity of encountering expired state where it was renewed by a sequence in the same compound, and the possibility of a COMPOUND returning when there is not much time left in the lease. a) Lease is renewed on SEQUENCE b) Lease is renewed upon termination of COMPOUND c) A lease is not allowed to expire if there is any currently executing COMPOUND for any associated session. I think this would make it easier to reason about the protocol since any COMPOUND will execute entirely within a single lease epoch (defined as the period between creation/revival of a lease and lease expiration). Note however that you could still get NFS4ERR_EXPIRED in response to SEQUENCE/PUTFH/READ, just as could have happened in v4.0 in response to SEQUENCE/READ, but only when lock was lost as a result of lease expiration before the SEQUENCE happened and so in this case (for v4.1) the sequence status bits will indicate the results of the lease expiration event. -----Original Message----- From: Mike Eisler [mailto:email2mre-ietf@yahoo.com]=20 Sent: Wednesday, September 05, 2007 10:49 AM To: nfsv4@ietf.org Subject: RE: [nfsv4] Create session and lease renewal --- "Noveck, Dave" wrote: > > Not just creating state. Say I have: > > > > SEQUENCE, PUTFH, READ > > > > In NFSv4.0 we'd have just had: > > > > PUTFH, READ > > > > and in NFSv4.0 the READ, because it took a stateid, renewed the > lease. >=20 > True but we are not under any obligation that I can see to renew the > lease under all circumstances under which it would have been renewed > in > v4.0.=20 So if the lease attribute is 60 seconds, at time T SEQUENCE is processed, and at time T+61 the READ is processed, I lose my lease, my locks, and the READ fails? It should be made clear in the spec if that's what we really want to design. I think that when this situation comes up, customers will be irate at the insult (lost lease) to injury (because a server takes 61 seconds to process a PUTFH ... it could happen if a file system is offline, or LDAP is offline during the export policy check) and server implementors will rapidly adopt my model. I know that realistically NFS4ERR_DELAY will be returned if PUTFH really takes 61 seconds, but doesn't detract much my point. If the server aborts PUTFH after say 20 seconds, then unless COMPOUND renews the lease at the end of processing, the client has just 40 seconds of lease left. So because the server, which is already slow, fails to renew the lease, the client has to send SEQUENCE more frequently, adding more load to the server, and exacerbating the situation. > > Wouldn't be simpler to state that the lease is renewed at=20 > > SEQUENCE, and MUST NOT expire during the processing of the=20 > > remaining operations in the COMPOUND, and MUST be renewed=20 > > after the COMPOUND is executed.=20 >=20 > It is simpler to say, but I'm not sure that the resulting protocol is > simpler (or even as simple). So you could have a lease renewed (at > SEQUENCE), the lease expire, have a lock created with a lease that is I wrote: "MUST NOT expire during processing of the remaining COMPOUND". If the lease is renewed at SEQUENCE, under my suggestion it expires no sooner than the wall clock time at when the COMPOUND completes plus the lease attribute. We already require the server to associate the sessionid (and the implied client ID) of the SEQUENCE with all remaining operations in the COMPOUND, and we already require that a COMPOUND have just one SEQUENCE. Further requring the timer that is the lease to be reset to the value of the lease attribute, when SEQUENCE is seen, then paused(i.e. not flow backwards) during the COMPOUND processing and then resuming the timer when the processing stops is really a small thing by comparison.=20 > expired (so maybe it vanishes as soon it is created?) and then the > lease > is revived when the COMPOUND ends. I'd like not to have describe all > the possibilities that that raises. I think it is simpler if > creation > of state is a lease-renewing/reviving event. >=20 > -----Original Message----- > From: Mike Eisler [mailto:email2mre-ietf@yahoo.com]=20 > Sent: Wednesday, September 05, 2007 9:19 AM > To: nfsv4@ietf.org > Subject: RE: [nfsv4] Create session and lease renewal >=20 >=20 > --- "Noveck, Dave" wrote: >=20 >=20 > > So given all that, I'm thinking that as part of the update for the=20 > > locking review, we say the lease is renewed at SEQUENCE and at any=20 > > operation that creates state and that having a session means you do >=20 > > have a piece of state. So the answer to whether CREATE_SESSION=20 > > renews/creates a lease would be "Yes". > >=20 > > So if there is no objection, that's what I'm going to do.=20 >=20 > Not just creating state. Say I have: >=20 > SEQUENCE, PUTFH, READ >=20 > In NFSv4.0 we'd have just had: >=20 > PUTFH, READ >=20 > and in NFSv4.0 the READ, because it took a stateid, renewed the > lease.=20 >=20 > Wouldn't be simpler to state that the lease is renewed at SEQUENCE, > and > MUST NOT expire during the processing of the remaining operations in > the > COMPOUND, and MUST be renewed after the COMPOUND is executed. >=20 >=20 >=20 > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 >=20 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:38:50 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwwE-0000Ip-GR; Wed, 05 Sep 2007 11:37:06 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwwC-0000GA-Jy for nfsv4@ietf.org; Wed, 05 Sep 2007 11:37:04 -0400 Received: from gw-colo-pa.panasas.com ([66.238.117.130] helo=cassoulet.panasas.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISwwB-0006UE-7V for nfsv4@ietf.org; Wed, 05 Sep 2007 11:37:04 -0400 Received: from riverside.int.panasas.com (localhost.localdomain [127.0.0.1]) by cassoulet.panasas.com (8.13.1/8.13.1) with ESMTP id l85Fb2rM013253; Wed, 5 Sep 2007 11:37:02 -0400 Received: from 172.17.1.41 ([172.17.1.41] helo=riverside.int.panasas.com) by ASSP-nospam; 5 Sep 2007 11:37:02 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [nfsv4] Create session and lease renewal Date: Wed, 5 Sep 2007 11:37:02 -0400 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nfsv4] Create session and lease renewal Thread-Index: Acfvzpph0Uo6pVvMTJe5OOjgSssQzAAALq7v References: <390555.61323.qm@web31812.mail.mud.yahoo.com> <969BB655-0D7E-40AE-8634-1AEEFE055224@sun.com> From: "Halevy, Benny" To: "Spencer Shepler" , X-Spam-Score: 0.0 (/) X-Scan-Signature: ded6070f7eed56e10c4f4d0d5043d9c7 Cc: nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org I agree too. =20 Back to the original question... Since CREATE_SESSION specifies the clientid it can renew the lease the same way SEQUENCE does. Do we want to special case it in case the clientid is already confirmed and the compound does not start with SEQUENCE? =20 Similarly this question could be asked with regards to EXCHANGE_ID=20 with EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and BIND_CONN_TO_SESSION. =20 Like I said before the lease can renewed whenever the clientid is = referenced, either directly or via the sessionid and be implicitly extended = throughout the life time of the COMPOUND as Mike proposed. =20 By the way, the spec does not say whether it is allowed to follow CREATE_SESSION with other operations in the compound. Theoretically I think that even if SEQUENCE precedes CREATE_SESSION such operations could be executed in the=20 context of the existing session however that seems to be confusing and I'd vote to explicitly disallow any other operation to follow CREATE_SESSION. =20 Benny ________________________________ From: Spencer Shepler [mailto:Spencer.Shepler@Sun.COM] Sent: Wed 2007-09-05 18:05 To: email2mre-ietf@yahoo.com Cc: nfsv4@ietf.org Subject: Re: [nfsv4] Create session and lease renewal I agree with this assessment, Mike. It is easier to understand even in the environment that COMPOUND is not atomic in nature. The COMPOUND is process as a whole and for very odd, edge case reasons, it may take a very long time (longer than the lease). Spencer On Sep 5, 2007, at 9:49 AM, Mike Eisler wrote: > > --- "Noveck, Dave" wrote: > >>> Not just creating state. Say I have: >>> >>> SEQUENCE, PUTFH, READ >>> >>> In NFSv4.0 we'd have just had: >>> >>> PUTFH, READ >>> >>> and in NFSv4.0 the READ, because it took a stateid, renewed the >> lease. >> >> True but we are not under any obligation that I can see to renew the >> lease under all circumstances under which it would have been renewed >> in >> v4.0. > > So if the lease attribute is 60 seconds, at time T SEQUENCE is > processed, and at time T+61 the READ is processed, I lose my lease, > my locks, and the READ fails? > > It should be made clear in the spec if that's what we really want > to design. > > I think that when this situation comes up, customers will be irate > at the insult (lost lease) to injury (because a server takes 61 > seconds to process a PUTFH ... it could happen if a file system is > offline, or LDAP is offline during the export policy check) and server > implementors will rapidly > adopt my model. > > I know that realistically NFS4ERR_DELAY will be returned if PUTFH > really takes 61 seconds, but doesn't detract much my point. If the > server aborts PUTFH after say 20 seconds, then unless COMPOUND > renews the lease at the end of processing, the client has just > 40 seconds of lease left. So because the server, which is already > slow, fails to renew the lease, the client has to send SEQUENCE > more frequently, adding more load to the server, and exacerbating > the situation. > >>> Wouldn't be simpler to state that the lease is renewed at >>> SEQUENCE, and MUST NOT expire during the processing of the >>> remaining operations in the COMPOUND, and MUST be renewed >>> after the COMPOUND is executed. >> >> It is simpler to say, but I'm not sure that the resulting protocol is >> simpler (or even as simple). So you could have a lease renewed (at >> SEQUENCE), the lease expire, have a lock created with a lease that is > > I wrote: "MUST NOT expire during processing of the remaining > COMPOUND". If the lease is renewed at SEQUENCE, under my > suggestion it expires no sooner than the wall clock > time at when the COMPOUND > completes plus the lease attribute. > > We already require the server to associate the sessionid (and > the implied client ID) of the SEQUENCE > with all remaining operations in the COMPOUND, and we already require > that a COMPOUND have just one SEQUENCE. Further requring the timer > that is the lease to be reset to the value of the lease attribute,=20 > when > SEQUENCE is seen, then paused(i.e. not flow backwards) during > the COMPOUND processing and then resuming the timer when the=20 > processing > stops is really a small thing by comparison. > >> expired (so maybe it vanishes as soon it is created?) and then the >> lease >> is revived when the COMPOUND ends. I'd like not to have describe all >> the possibilities that that raises. I think it is simpler if >> creation >> of state is a lease-renewing/reviving event. >> >> -----Original Message----- >> From: Mike Eisler [mailto:email2mre-ietf@yahoo.com] >> Sent: Wednesday, September 05, 2007 9:19 AM >> To: nfsv4@ietf.org >> Subject: RE: [nfsv4] Create session and lease renewal >> >> >> --- "Noveck, Dave" wrote: >> >> >>> So given all that, I'm thinking that as part of the update for the >>> locking review, we say the lease is renewed at SEQUENCE and at any >>> operation that creates state and that having a session means you do >> >>> have a piece of state. So the answer to whether CREATE_SESSION >>> renews/creates a lease would be "Yes". >>> >>> So if there is no objection, that's what I'm going to do. >> >> Not just creating state. Say I have: >> >> SEQUENCE, PUTFH, READ >> >> In NFSv4.0 we'd have just had: >> >> PUTFH, READ >> >> and in NFSv4.0 the READ, because it took a stateid, renewed the >> lease. >> >> Wouldn't be simpler to state that the lease is renewed at SEQUENCE, >> and >> MUST NOT expire during the processing of the remaining operations in >> the >> COMPOUND, and MUST be renewed after the COMPOUND is executed. >> >> >> >> _______________________________________________ >> nfsv4 mailing list >> nfsv4@ietf.org >> https://www1.ietf.org/mailman/listinfo/nfsv4 >> > > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:42:27 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwzh-0002fN-Q5; Wed, 05 Sep 2007 11:40:41 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISwzg-0002fH-TE for nfsv4@ietf.org; Wed, 05 Sep 2007 11:40:40 -0400 Received: from web31803.mail.mud.yahoo.com ([68.142.207.66]) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1ISwzf-0006Y5-KX for nfsv4@ietf.org; Wed, 05 Sep 2007 11:40:40 -0400 Received: (qmail 72228 invoked by uid 60001); 5 Sep 2007 15:40:39 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=6klBRb/xpFXLSCUUFa1jroe+s1RbOWUQUicAwiKD7QREyXJT4sLLcpv7zUP9jPLL7HxSaxr8B7n7Diutp8BFG/72xDIyDIHGk9WE63cJ/330pYbsbZy9V1/7EilpWIAYwjl9rJTzcyNh2rD1hHYUCJ0h4eB3h11yFcdPU/Xv+6I=; X-YMail-OSG: dG2qTBMVM1mfLmzHXzPf7pY5v0AZ7GOinzXnENHlcpmcSVvwumjp4lTB4xfmw23psdXnGp.GXw-- Received: from [198.95.226.224] by web31803.mail.mud.yahoo.com via HTTP; Wed, 05 Sep 2007 08:40:38 PDT Date: Wed, 5 Sep 2007 08:40:38 -0700 (PDT) From: Mike Eisler Subject: RE: [nfsv4] Create session and lease renewal To: nfsv4@ietf.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <104212.71685.qm@web31803.mail.mud.yahoo.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: 93238566e09e6e262849b4f805833007 X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: email2mre-ietf@yahoo.com List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org We are in perfect agreement; I was apparently not being clear. --- "Noveck, Dave" wrote: > So here is a model that I think addresses my issue, the possiblity of > creating a lock while a lease is expired, and your issues, the > possiblity of encountering expired state where it was renewed by a > sequence in the same compound, and the possibility of a COMPOUND > returning when there is not much time left in the lease. > > a) Lease is renewed on SEQUENCE > > b) Lease is renewed upon termination of COMPOUND > > c) A lease is not allowed to expire if there is any currently > executing > COMPOUND for any associated session. > > I think this would make it easier to reason about the protocol since > any > COMPOUND will execute entirely within a single lease epoch (defined > as > the period between creation/revival of a lease and lease expiration). _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:45:56 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISx33-0005bt-P2; Wed, 05 Sep 2007 11:44:09 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISx32-0005U0-4n for nfsv4@ietf.org; Wed, 05 Sep 2007 11:44:08 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ISx30-0006bz-OT for nfsv4@ietf.org; Wed, 05 Sep 2007 11:44:08 -0400 X-IronPort-AV: E=Sophos;i="4.20,211,1186383600"; d="scan'208";a="100455205" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 05 Sep 2007 08:44:06 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l85Fi5K0010837; Wed, 5 Sep 2007 08:44:05 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 08:44:05 -0700 Received: from exnane01.hq.netapp.com ([10.97.0.61]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 08:44:05 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [nfsv4] Create session and lease renewal Date: Wed, 5 Sep 2007 11:44:03 -0400 Message-ID: In-Reply-To: <104212.71685.qm@web31803.mail.mud.yahoo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nfsv4] Create session and lease renewal Thread-Index: Acfv0y1OQ416oDM6R/q963bIFnJSPQAAFEyw From: "Noveck, Dave" To: , X-OriginalArrivalTime: 05 Sep 2007 15:44:05.0090 (UTC) FILETIME=[96D30C20:01C7EFD3] X-Spam-Score: -4.0 (----) X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1 Cc: X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org I didn't read your note carefully enough. Sorry. -----Original Message----- From: Mike Eisler [mailto:email2mre-ietf@yahoo.com]=20 Sent: Wednesday, September 05, 2007 11:41 AM To: nfsv4@ietf.org Subject: RE: [nfsv4] Create session and lease renewal We are in perfect agreement; I was apparently not being clear. --- "Noveck, Dave" wrote: > So here is a model that I think addresses my issue, the possiblity of > creating a lock while a lease is expired, and your issues, the > possiblity of encountering expired state where it was renewed by a > sequence in the same compound, and the possibility of a COMPOUND > returning when there is not much time left in the lease. >=20 > a) Lease is renewed on SEQUENCE >=20 > b) Lease is renewed upon termination of COMPOUND >=20 > c) A lease is not allowed to expire if there is any currently > executing > COMPOUND for any associated session. >=20 > I think this would make it easier to reason about the protocol since > any > COMPOUND will execute entirely within a single lease epoch (defined > as > the period between creation/revival of a lease and lease expiration). _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 11:52:18 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISx9H-0001uX-Bq; Wed, 05 Sep 2007 11:50:35 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ISx9G-0001uL-7g for nfsv4@ietf.org; Wed, 05 Sep 2007 11:50:34 -0400 Received: from web31802.mail.mud.yahoo.com ([68.142.207.65]) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1ISx9E-0006lm-U4 for nfsv4@ietf.org; Wed, 05 Sep 2007 11:50:34 -0400 Received: (qmail 32045 invoked by uid 60001); 5 Sep 2007 15:50:32 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=TqliGdl8lkTJV5gI3YDn/mMN2b7c7jOt6spayrfINnjfIbtMU4PqqWTPJLYhNhWXRf3JQp0Na9IqPqD9ji7TDhYR0jT5lRLNtqYu630uzP7XL6V6MivoGQ5sq/9Z2zMsiggQa3H5fa94lc620cZ9SI1M9+9rgjWg0EY8X1CSyU0=; X-YMail-OSG: qiFHufYVM1mgA74jHLpA6zwg45vPy8aZQIUhmJoFLfZXtGn9GhrG7IjPpDkpo0vehJw3dLXsHSR9rWLN36MxOyVbNKi1QmVthgE_Y1J_lB5GdG4- Received: from [198.95.226.224] by web31802.mail.mud.yahoo.com via HTTP; Wed, 05 Sep 2007 08:50:32 PDT Date: Wed, 5 Sep 2007 08:50:32 -0700 (PDT) From: Mike Eisler Subject: RE: [nfsv4] Create session and lease renewal To: "Halevy, Benny" , Spencer Shepler , email2mre-ietf@yahoo.com In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <374061.31979.qm@web31802.mail.mud.yahoo.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: f884eb1d4ec5a230688d7edc526ea665 Cc: nfsv4@ietf.org X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: email2mre-ietf@yahoo.com List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org --- "Halevy, Benny" wrote: > Back to the original question... > Since CREATE_SESSION specifies the clientid it can > renew the lease the same way SEQUENCE does. > Do we want to special case it in case the clientid is already > confirmed and the compound does not start with SEQUENCE? My opinion is no, since the special case is not necessary. If you want a CREATE_SESSION to renew an existing client ID's lease, then prefix CREATE_SESSION with a SEQUENCE for another sessionid with the same client ID. > > Similarly this question could be asked with regards to EXCHANGE_ID > with EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and > BIND_CONN_TO_SESSION. Ditto for the former; Prefix such an EXCHANGE_ID with SEQUENCE. For the latter, an argument for a special case to have BIND_CONN_TO_SESSION renew the lease is that a client might have lost all its connections on the session. The state protection model (selected when EXCHANGE_ID/CREATE_SESSION established and confirmed the client ID) might require BIND_CONN_TO_SESSION. In that situation, the client is in peril of losing its lease, so the sooner it can do that, the better. > By the way, the spec does not say whether it is allowed > to follow CREATE_SESSION with other operations OK, we need to fix that. > in the compound. Theoretically I think that even if SEQUENCE > precedes > CREATE_SESSION such operations could be executed in the > context of the existing session however that seems to be confusing > and I'd vote to explicitly disallow any other operation to follow > CREATE_SESSION. If SEQUENCE precedes CREATE_SESSION, it seems unnatural to disallow subsequent operations. If SEQUENCE does not precede CREATE_SESSION, the set of operations that could possibly follow CREATE_SESSION is limited. Regardless, I think it is reasonable to have a blanket rule that says unless SEQUENCE starts the COMPOUND, COMPOUND can have just one operation. > > Benny > > ________________________________ > > From: Spencer Shepler [mailto:Spencer.Shepler@Sun.COM] > Sent: Wed 2007-09-05 18:05 > To: email2mre-ietf@yahoo.com > Cc: nfsv4@ietf.org > Subject: Re: [nfsv4] Create session and lease renewal > > > > > I agree with this assessment, Mike. > > It is easier to understand even in the environment that > COMPOUND is not atomic in nature. The COMPOUND is process > as a whole and for very odd, edge case reasons, it may take > a very long time (longer than the lease). > > Spencer > > On Sep 5, 2007, at 9:49 AM, Mike Eisler wrote: > > > > > --- "Noveck, Dave" wrote: > > > >>> Not just creating state. Say I have: > >>> > >>> SEQUENCE, PUTFH, READ > >>> > >>> In NFSv4.0 we'd have just had: > >>> > >>> PUTFH, READ > >>> > >>> and in NFSv4.0 the READ, because it took a stateid, renewed the > >> lease. > >> > >> True but we are not under any obligation that I can see to renew > the > >> lease under all circumstances under which it would have been > renewed > >> in > >> v4.0. > > > > So if the lease attribute is 60 seconds, at time T SEQUENCE is > > processed, and at time T+61 the READ is processed, I lose my lease, > > my locks, and the READ fails? > > > > It should be made clear in the spec if that's what we really want > > to design. > > > > I think that when this situation comes up, customers will be irate > > at the insult (lost lease) to injury (because a server takes 61 > > seconds to process a PUTFH ... it could happen if a file system is > > offline, or LDAP is offline during the export policy check) and > server > > implementors will rapidly > > adopt my model. > > > > I know that realistically NFS4ERR_DELAY will be returned if PUTFH > > really takes 61 seconds, but doesn't detract much my point. If the > > server aborts PUTFH after say 20 seconds, then unless COMPOUND > > renews the lease at the end of processing, the client has just > > 40 seconds of lease left. So because the server, which is already > > slow, fails to renew the lease, the client has to send SEQUENCE > > more frequently, adding more load to the server, and exacerbating > > the situation. > > > >>> Wouldn't be simpler to state that the lease is renewed at > >>> SEQUENCE, and MUST NOT expire during the processing of the > >>> remaining operations in the COMPOUND, and MUST be renewed > >>> after the COMPOUND is executed. > >> > >> It is simpler to say, but I'm not sure that the resulting protocol > is > >> simpler (or even as simple). So you could have a lease renewed > (at > >> SEQUENCE), the lease expire, have a lock created with a lease that > is > > > > I wrote: "MUST NOT expire during processing of the remaining > > COMPOUND". If the lease is renewed at SEQUENCE, under my > > suggestion it expires no sooner than the wall clock > > time at when the COMPOUND > > completes plus the lease attribute. > > > > We already require the server to associate the sessionid (and > > the implied client ID) of the SEQUENCE > > with all remaining operations in the COMPOUND, and we already > require > > that a COMPOUND have just one SEQUENCE. Further requring the timer > > that is the lease to be reset to the value of the lease attribute, > > when > > SEQUENCE is seen, then paused(i.e. not flow backwards) during > > the COMPOUND processing and then resuming the timer when the > > processing > > stops is really a small thing by comparison. > > > >> expired (so maybe it vanishes as soon it is created?) and then the > >> lease > >> is revived when the COMPOUND ends. I'd like not to have describe > all > >> the possibilities that that raises. I think it is simpler if > >> creation > >> of state is a lease-renewing/reviving event. > >> > >> -----Original Message----- > >> From: Mike Eisler [mailto:email2mre-ietf@yahoo.com] > >> Sent: Wednesday, September 05, 2007 9:19 AM > >> To: nfsv4@ietf.org > >> Subject: RE: [nfsv4] Create session and lease renewal > >> > >> > >> --- "Noveck, Dave" wrote: > >> > >> > >>> So given all that, I'm thinking that as part of the update for > the > >>> locking review, we say the lease is renewed at SEQUENCE and at > any > >>> operation that creates state and that having a session means you > do > >> > >>> have a piece of state. So the answer to whether CREATE_SESSION > >>> renews/creates a lease would be "Yes". > >>> > >>> So if there is no objection, that's what I'm going to do. > >> > >> Not just creating state. Say I have: > >> > >> SEQUENCE, PUTFH, READ > >> > >> In NFSv4.0 we'd have just had: > >> > >> PUTFH, READ > >> > >> and in NFSv4.0 the READ, because it took a stateid, renewed the > >> lease. > >> > >> Wouldn't be simpler to state that the lease is renewed at > SEQUENCE, > >> and > >> MUST NOT expire during the processing of the remaining operations > in > >> the > >> COMPOUND, and MUST be renewed after the COMPOUND is executed. > >> > >> > >> > >> _______________________________________________ > >> nfsv4 mailing list > >> nfsv4@ietf.org > >> https://www1.ietf.org/mailman/listinfo/nfsv4 > >> > > > > > > _______________________________________________ > > nfsv4 mailing list > > nfsv4@ietf.org > > https://www1.ietf.org/mailman/listinfo/nfsv4 > > > _______________________________________________ > nfsv4 mailing list > nfsv4@ietf.org > https://www1.ietf.org/mailman/listinfo/nfsv4 > > > _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Wed Sep 05 16:17:23 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IT1Hl-0003jU-TD; Wed, 05 Sep 2007 16:15:37 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IT1Hh-0003f2-2z; Wed, 05 Sep 2007 16:15:33 -0400 Received: from ns4.neustar.com ([156.154.24.139]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IT1Hg-0000xl-MW; Wed, 05 Sep 2007 16:15:33 -0400 Received: from stiedprstage1.ietf.org (stiedprstage1.va.neustar.com [10.31.47.10]) by ns4.neustar.com (Postfix) with ESMTP id 948342AC64; Wed, 5 Sep 2007 20:15:02 +0000 (GMT) Received: from ietf by stiedprstage1.ietf.org with local (Exim 4.43) id 1IT1HC-00041u-BV; Wed, 05 Sep 2007 16:15:02 -0400 Content-Type: Multipart/Mixed; Boundary="NextPart" Mime-Version: 1.0 To: i-d-announce@ietf.org From: Internet-Drafts@ietf.org Message-Id: Date: Wed, 05 Sep 2007 16:15:02 -0400 X-Spam-Score: 0.0 (/) X-Scan-Signature: 31247fb3be228bb596db9127becad0bc Cc: nfsv4@ietf.org Subject: [nfsv4] I-D ACTION:draft-ietf-nfsv4-pnfs-obj-04.txt X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Network File System Version 4 Working Group of the IETF. Title : Object-based pNFS Operations Author(s) : B. Halevy, et al. Filename : draft-ietf-nfsv4-pnfs-obj-04.txt Pages : 29 Date : 2007-9-5 This Internet-Draft provides a description of the object-based pNFS extension for NFSv4. This is a companion to the main pnfs specification in the NFSv4 Minor Version 1 Internet Draft, which is currently draft-ietf-nfsv4-minorversion1-13.txt. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-pnfs-obj-04.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request@ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-nfsv4-pnfs-obj-04.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-nfsv4-pnfs-obj-04.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2007-9-5151628.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-nfsv4-pnfs-obj-04.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-nfsv4-pnfs-obj-04.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2007-9-5151628.I-D@ietf.org> --OtherAccess-- --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 --NextPart-- From nfsv4-bounces@ietf.org Wed Sep 05 16:20:59 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IT1LG-00055o-Lk; Wed, 05 Sep 2007 16:19:14 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IT0M9-0005Zy-Do for nfsv4@ietf.org; Wed, 05 Sep 2007 15:16:05 -0400 Received: from gw-colo-pa.panasas.com ([66.238.117.130] helo=cassoulet.panasas.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IT0M6-0003NC-4l for nfsv4@ietf.org; Wed, 05 Sep 2007 15:16:05 -0400 Received: from riverside.int.panasas.com (localhost.localdomain [127.0.0.1]) by cassoulet.panasas.com (8.13.1/8.13.1) with ESMTP id l85JFwCJ020578 for ; Wed, 5 Sep 2007 15:15:58 -0400 Received: from 172.17.1.41 ([172.17.1.41] helo=riverside.int.panasas.com) by ASSP-nospam; 5 Sep 2007 15:15:58 -0400 Received: from [172.17.28.128] ([172.17.28.128]) by riverside.int.panasas.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 5 Sep 2007 15:15:51 -0400 Message-ID: <46DF0065.7090808@panasas.com> Date: Wed, 05 Sep 2007 22:15:49 +0300 From: Benny Halevy User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: nfsv4@ietf.org Content-Type: multipart/mixed; boundary="------------010303050605070809060207" X-OriginalArrivalTime: 05 Sep 2007 19:15:51.0674 (UTC) FILETIME=[2C89E5A0:01C7EFF1] X-Spam-Score: 1.8 (+) X-Scan-Signature: 446d68a75b398ee3f32d8edb697d0531 X-Mailman-Approved-At: Wed, 05 Sep 2007 16:19:12 -0400 Subject: [nfsv4] Fwd: draft-ietf-nfsv4-pnfs-obj-04 X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org This is a multi-part message in MIME format. --------------010303050605070809060207 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I submitted draft-ietf-nfsv4-pnfs-obj-04. Please see summary of changes below. Benny -------- Original Message -------- Subject: draft-ietf-nfsv4-pnfs-obj-04 Date: Wed, 05 Sep 2007 22:13:20 +0300 From: Benny Halevy To: internet-drafts@ietf.org CC: Benny Halevy References: <45EBFF89.4040305@panasas.com> Attached is an updated I-D, draft-ietf-nfsv4-pnfs-obj-04.txt and its corresponding source file, draft-ietf-nfsv4-pnfs-obj-04.xml. It updates the nfsv4 working group I-D, draft-ietf-nfsv4-pnfs-obj-03.txt The new version has several editorial changes and clarifications as well these technical changes: - Added optional System ID to pnfs_osd_deviceaddr4. Clarified its use in the system and how it is related to the OSD name. - Added cap_key_sec to pnfs_osd_object_cred4 specifying the method used to secure the capability key. Specified the different methods. - Broke the opaque pnfs_osd_object_cred4.credential into two opaque fields: capability_key and capability since the pnfs client must be able to parse the internal structure of the credential. - Reorganized and clarified pnfs_osd_errno4 error codes. Added PNFS_OSD_ERR_RESOURCE. - Specified object-based layout specific mask flags for CB_RECALL_ALL. - Added Client Fencing section. - Added paragraph to Security Considerations describing the method used to enforce the server's access control policy and the possibility of rejecting LAYOUTGET with NFS4ERR_ACCESS. - Disallowed the used of NOSEC OSD security method for anything but the first OSD GET ATTRIBUTE the might be needed to read the OSD System ID. - Separated the protocol privacy requirements as a sub-section in Security Considerations. - Corrected Revoking Capabilities section. -- Benny Halevy Software Architect Tel/Fax: +972-3-647-8340 Mobile: +972-54-802-8340 US: +1-412-203-3187 bhalevy@panasas.com Panasas, Inc. Leader in Parallel Storage www.panasas.com --------------010303050605070809060207 Content-Type: text/plain; name="draft-ietf-nfsv4-pnfs-obj-04.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="draft-ietf-nfsv4-pnfs-obj-04.txt" NFSv4 B. Halevy Internet-Draft B. Welch Intended status: Standards Track J. Zelenka Expires: March 8, 2008 Panasas September 5, 2007 Object-based pNFS Operations draft-ietf-nfsv4-pnfs-obj-04 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on March 8, 2008. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This Internet-Draft provides a description of the object-based pNFS extension for NFSv4. This is a companion to the main pnfs specification in the NFSv4 Minor Version 1 Internet Draft, which is currently draft-ietf-nfsv4-minorversion1-13.txt. Halevy, et al. Expires March 8, 2008 [Page 1] Internet-Draft pnfs objects September 2007 Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [1]. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Object Storage Device Addressing and Discovery . . . . . . . . 4 2.1. pnfs_osd_addr_type4 . . . . . . . . . . . . . . . . . . . 5 2.2. pnfs_osd_deviceaddr4 . . . . . . . . . . . . . . . . . . . 6 3. Object-Based Layout . . . . . . . . . . . . . . . . . . . . . 6 3.1. pnfs_osd_layout4 . . . . . . . . . . . . . . . . . . . . . 7 3.1.1. pnfs_osd_objid4 . . . . . . . . . . . . . . . . . . . 7 3.1.2. pnfs_osd_version4 . . . . . . . . . . . . . . . . . . 8 3.1.3. pnfs_osd_object_cred4 . . . . . . . . . . . . . . . . 8 3.1.4. pnfs_osd_raid_algorithm4 . . . . . . . . . . . . . . . 10 3.1.5. pnfs_osd_data_map4 . . . . . . . . . . . . . . . . . . 10 3.2. Data Mapping Schemes . . . . . . . . . . . . . . . . . . . 11 3.2.1. Simple Striping . . . . . . . . . . . . . . . . . . . 11 3.2.2. Nested Striping . . . . . . . . . . . . . . . . . . . 12 3.2.3. Mirroring . . . . . . . . . . . . . . . . . . . . . . 13 3.3. RAID Algorithms . . . . . . . . . . . . . . . . . . . . . 14 3.3.1. PNFS_OSD_RAID_0 . . . . . . . . . . . . . . . . . . . 14 3.3.2. PNFS_OSD_RAID_4 . . . . . . . . . . . . . . . . . . . 14 3.3.3. PNFS_OSD_RAID_5 . . . . . . . . . . . . . . . . . . . 15 3.3.4. PNFS_OSD_RAID_PQ . . . . . . . . . . . . . . . . . . . 15 3.3.5. RAID Usage and implementation notes . . . . . . . . . 16 4. Object-Based Layout Update . . . . . . . . . . . . . . . . . . 16 4.1. pnfs_osd_layoutupdate4 . . . . . . . . . . . . . . . . . . 16 4.1.1. pnfs_osd_deltaspaceused4 . . . . . . . . . . . . . . . 17 4.1.2. pnfs_osd_errno4 . . . . . . . . . . . . . . . . . . . 17 4.1.3. pnfs_osd_ioerr4 . . . . . . . . . . . . . . . . . . . 18 5. Object-Based Creation Layout Hint . . . . . . . . . . . . . . 19 5.1. pnfs_osd_layouthint4 . . . . . . . . . . . . . . . . . . . 19 6. Layout Segments . . . . . . . . . . . . . . . . . . . . . . . 20 6.1. CB_LAYOUTRECALL and LAYOUTRETURN . . . . . . . . . . . . . 20 6.2. LAYOUTCOMMIT . . . . . . . . . . . . . . . . . . . . . . . 21 7. Recalling Layouts . . . . . . . . . . . . . . . . . . . . . . 21 7.1. CB_RECALL_ANY . . . . . . . . . . . . . . . . . . . . . . 22 8. Client Fencing . . . . . . . . . . . . . . . . . . . . . . . . 22 9. Security Considerations . . . . . . . . . . . . . . . . . . . 23 9.1. OSD Security Data Types . . . . . . . . . . . . . . . . . 24 9.2. The OSD Security Protocol . . . . . . . . . . . . . . . . 24 9.3. Protocol Privacy Requirements . . . . . . . . . . . . . . 25 9.4. Revoking Capabilities . . . . . . . . . . . . . . . . . . 26 Halevy, et al. Expires March 8, 2008 [Page 2] Internet-Draft pnfs objects September 2007 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 11.1. Normative References . . . . . . . . . . . . . . . . . . . 27 11.2. Informative References . . . . . . . . . . . . . . . . . . 27 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 27 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 Intellectual Property and Copyright Statements . . . . . . . . . . 29 Halevy, et al. Expires March 8, 2008 [Page 3] Internet-Draft pnfs objects September 2007 1. Introduction In pNFS, the file server returns typed layout structures that describe where file data is located. There are different layouts for different storage systems and methods of arranging data on storage devices. This document describes the layouts used with object-based storage devices (OSD) that are accessed according to the iSCSI/OSD storage protocol standard (SNIA T10/1355-D [2]). An "object" is a container for data and attributes, and files are stored in one or more objects. The OSD protocol specifies several operations on objects, including READ, WRITE, FLUSH, GET ATTRIBUTES, SET ATTRIBUTES, CREATE and DELETE. However, using the object-based layout the client only uses the READ, WRITE, GET ATTRIBUTES and FLUSH commands. The other commands are only used by the pNFS server. An object-based layout for pNFS includes object identifiers, capabilities that allow clients to READ or WRITE those objects, and various parameters that control how file data is striped across their component objects. The OSD protocol has a capability-based security scheme that allows the pNFS server to control what operations and what objects can be used by clients. This scheme is described in more detail in the Security Considerations section (Section 9). 2. Object Storage Device Addressing and Discovery Data operations to an OSD require the client to know the "address" of each OSD's root object. The root object is synonymous with SCSI logical unit. The client specifies SCSI logical units to its SCSI stack using a representation local to the client. Because these representations are local, GETDEVICEINFO must return information that can be used by the client to select the correct local representation. In the block world, a set offset (logical block number or track/ sector) contains a disk label. This label identifies the disk uniquely. In contrast, an OSD has a standard set of attributes on its root object. For device identification purposes the OSD System ID (root information attribute number 3) and/or OSD Name (root information attribute number 9) are used as the label. These appear in the pnfs_osd_deviceaddr4 type below under the "systemid" and "osdname" fields. In some situations, SCSI target discovery may need to be driven based on information contained in the GETDEVICEINFO response. One example of this is iSCSI targets that are not known to the client until a layout has been requested. Eventually iSCSI will adopt ANSI T10 SAM-3, at which time the World Wide Name (WWN aka, EUI-64/EUI-128) Halevy, et al. Expires March 8, 2008 [Page 4] Internet-Draft pnfs objects September 2007 naming conventions can be specified. In addition, Fibre Channel (FC) SCSI targets have a unique WWN. Although these FC targets have already been discovered, some implementations may want to specify the WWN in addition to the label. This information appears as the "target" and "lun" fields in the pnfs_osd_deviceaddr4 type described below. The systemid is used by the client, along with the object credential to sign each request with the request integrity check value. This method protects the client from unintentionally accessing a device if the device address mapping was changed (or revoked). The server computes the capability_key using its own view of the systemid associated with the respective deviceid present in the credential. If the client's view of the deviceid mapping is stale, the client will use the wrong systemid (which must be system-wide unique) and the I/O request to the OSD will fail to pass the integrity check verification. To recover from this condition the client should report the error via LAYOUTCOMMIT, return the layout using LAYOUTRETURN, and invalidate all the device address mappings associated with this layout. The client can then ask for a new layout if it wishes using LAYOUTGET and resolve the referenced deviceids using GETDEVICEINFO or GETDEVICELIST. The server MUST provide either the systemid, the OSD name, or both. When the OSD name is present the client SHOULD get the root information attributes whenever it establishes communication with the OSD and verify that the OSD name it got from the OSD matches the one sent by the metadata server. If the systemid was not given by the server it MUST be taken from the OSD-provided attribute; note that in this case the OSD GET ATTRIBUTES operation must be performed with the NOSEC security method. 2.1. pnfs_osd_addr_type4 The following enum specifies the manner in which a scsi target can be specified. The target can be specified as a network address, as an Internet Qualified Name (IQN), or by the World-Wide Name (WWN) of the target. enum pnfs_obj_addr_type4 { OBJ_TARGET_NETADDR = 1, OBJ_TARGET_IQN = 2, OBJ_TARGET_WWN = 3 }; Halevy, et al. Expires March 8, 2008 [Page 5] Internet-Draft pnfs objects September 2007 2.2. pnfs_osd_deviceaddr4 The specification for an object device address is as follows: struct pnfs_osd_deviceaddr4 { union target switch (pnfs_osd_addr_type4 type) { case OBJ_TARGET_NETADDR: pnfs_netaddr4 netaddr; case OBJ_TARGET_IQN: string iqn<>; case OBJ_TARGET_WWN: string wwn<>; default: void; }; uint64_t lun; opaque systemid<>; opaque osdname<>; }; 3. Object-Based Layout The layout4 type is defined in the NFSv4.1 draft [6] as follows: enum layouttype4 { LAYOUT4_NFSV4_1_FILES = 1, LAYOUT4_OSD2_OBJECTS = 2, LAYOUT4_BLOCK_VOLUME = 3 }; struct layout_content4 { layouttype4 loc_type; opaque loc_body<>; }; struct layout4 { offset4 lo_offset; length4 lo_length; layoutiomode4 lo_iomode; layout_content4 lo_content; }; This document defines structure associated with the layouttype4 Halevy, et al. Expires March 8, 2008 [Page 6] Internet-Draft pnfs objects September 2007 value, LAYOUT4_OSD2_OBJECTS. The NFSv4.1 draft [6] specifies the loc_body structure as an XDR type "opaque". The opaque layout is uninterpreted by the generic pNFS client layers, but obviously must be interpreted by the object-storage layout driver. This document defines the structure of this opaque value, pnfs_osd_layout4. 3.1. pnfs_osd_layout4 struct pnfs_osd_layout4 { pnfs_osd_data_map4 map; pnfs_osd_object_cred4 components<>; }; The pnfs_osd_layout4 structure specifies a layout over a set of component objects. The components field is an array of object identifiers and security credentials that grant access to each object. The organization of the data is defined by the pnfs_osd_data_map4 type that specifies how the file's data is mapped onto the component objects (i.e., the striping pattern). The data placement algorithm that maps file data onto component objects assume that each component object occurs exactly once in the array of components. Therefore, component objects MUST appear in the component array only once. Note that the layout depends on the file size, which the client learns from the generic return parameters of LAYOUTGET, by doing GETATTR commands to the metadata server. The client uses the file size to decide if it should fill holes with zeros, or return a short read. Striping patterns can cause cases where component objects are shorter than other components because a hole happens to correspond to the last part of the component object. 3.1.1. pnfs_osd_objid4 An object is identified by a number, somewhat like an inode number. The object storage model has a two level scheme, where the objects within an object storage device are grouped into partitions. struct pnfs_osd_objid4 { deviceid4 device_id; uint64_t partition_id; uint64_t object_id; }; The pnfs_osd_objid4 type is used to identify an object within a partition on a specified object storage device. "device_id" selects the object storage device from the set of available storage devices. The device is identified with the deviceid4 type, which is an index Halevy, et al. Expires March 8, 2008 [Page 7] Internet-Draft pnfs objects September 2007 into addressing information about that device returned by the GETDEVICELIST and GETDEVICEINFO pnfs operations. Within an OSD, a partition is identified with a 64-bit number, "partition_id". Within a partition, an object is identified with a 64-bit number, "object_id". Creation and management of partitions is outside the scope of this standard, and is a facility provided by the object storage file system. 3.1.2. pnfs_osd_version4 enum pnfs_osd_version4 { PNFS_OSD_MISSING = 0, PNFS_OSD_VERSION_1 = 1, PNFS_OSD_VERSION_2 = 2 }; The osd_version is used to indicate the OSD protocol version or whether an object is missing (i.e., unavailable). Some layout schemes encode redundant information and can compensate for missing components, but the data placement algorithm needs to know what parts are missing. At this time the OSD standard is at version 1.0, and we anticipate a version 2.0 of the standard ((SNIA T10/1729-D [7])). The second generation OSD protocol has additional proposed features to support more robust error recovery, snapshots, and byte-range capabilities. Therefore, the OSD version is explicitly called out in the information returned in the layout. (This information can also be deduced by looking inside the capability type at the format field, which is the first byte. The format value is 0x1 for an OSD v1 capability. However, it seems most robust to call out the version explicitly.) 3.1.3. pnfs_osd_object_cred4 enum pnfs_osd_cap_key_sec4 { PNFS_OSD_CAP_KEY_SEC_NONE = 0, PNFS_OSD_CAP_KEY_SEC_SSV = 1, }; struct pnfs_osd_object_cred4 { pnfs_osd_objid4 object_id; pnfs_osd_version4 osd_version; pnfs_osd_cap_key_sec4 cap_key_sec; opaque capability_key<>; opaque capability<>; }; Halevy, et al. Expires March 8, 2008 [Page 8] Internet-Draft pnfs objects September 2007 The pnfs_osd_object_cred4 structure is used to identify each component comprising the file. The object_id identifies the component object, the osd_version represents the osd protocol version, or whether that component is unavailable, and the capability and capability key, along with the systemid from the pnfs_osd_deviceaddr, provide the OSD security credentials needed to access that object. The cap_key_sec value denotes the method used to secure the capability_key (see Section 9.1 for more details). To comply with the OSD security requirements the capability key SHOULD be transferred securely to prevent eavesdropping (see Section 9). Therefore, a client SHOULD either issue the LAYOUTGET operation via RPCSEC_GSS with the privacy service or to previously establish an SSV for the sessions via the NFSv4.1 SET_SSV operation. The pnfs_osd_cap_key_sec4 type is used to identify the method used by the server to secure the capability key. o PNFS_OSD_CAP_KEY_SEC_NONE denotes that the capability_key is not encrypted in which case the client SHOULD issue the LAYOUTGET operation with RPCSEC_GSS with the privacy service or the NFSv4.1 transport should be secured by using methods that are external to NFSv4.1 like the use of IPSEC [8] for transporting the NFSV4.1 protocol. o PNFS_OSD_CAP_KEY_SEC_SSV denotes that the capability_key contents are encrypted using the SSV GSS context and the capability key as inputs to the GSS_Wrap() function (see [3]) with the conf_req_flag set to TRUE. The client MUST use the secret SSV key as part of the client's GSS context to decrypt the capability key using the value of the capability_key field as the input_message to the GSS_unwrap() function. Note that to prevent eavesdropping of the SSV key the client SHOULD issue SET_SSV via RPCSEC_GSS with the privacy service. The actual method chosen depends on whether the client established a SSV key with the server and whether it issued the LAYOUTGET operation with the RPCSEC_GSS privacy method. Naturally, if the client did not establish a SSV key via SET_SSV the server MUST use the PNFS_OSD_CAP_KEY_SEC_NONE method. Otherwise, if the LAYOUTGET operation was not issued with the RPCSEC_GSS privacy method the server SHOULD secure the capability_key with the PNFS_OSD_CAP_KEY_SEC_SSV method. The server MAY use the PNFS_OSD_CAP_KEY_SEC_SSV method also when the LAYOUTGET operation was issued with the RPCSEC_GSS privacy method. Halevy, et al. Expires March 8, 2008 [Page 9] Internet-Draft pnfs objects September 2007 3.1.4. pnfs_osd_raid_algorithm4 enum pnfs_osd_raid_algorithm4 { PNFS_OSD_RAID_0 = 1, PNFS_OSD_RAID_4 = 2, PNFS_OSD_RAID_5 = 3, PNFS_OSD_RAID_PQ = 4 /* Reed-Solomon P+Q */ }; pnfs_osd_raid_algorithm4 represents the data redundancy algorithm used to protect the file's contents. See Section 3.3 for more details. 3.1.5. pnfs_osd_data_map4 struct pnfs_osd_data_map4 { length4 stripe_unit; uint32_t group_width; uint32_t group_depth; uint32_t mirror_cnt; pnfs_osd_raid_algorithm4 raid_algorithm; }; The pnfs_osd_data_map4 structure parameterizes the algorithm that maps a file's contents over the component objects. Instead of limiting the system to simple striping scheme where loss of a single component object results in data loss, the map parameters support mirroring and more complicated schemes that protect against loss of a component object. The stripe_unit is the number of bytes placed on one component before advancing to the next one in the list of components. The number of bytes in a full stripe is stripe_unit times the number of components. In some raid schemes, a stripe includes redundant information (i.e., parity) that lets the system recover from loss or damage to a component object. The group_width and group_depth parameters allow a nested striping pattern. If there is no nesting, then group_width and group_depth MUST be zero. Otherwise, the group_width defines the width of a data stripe, and the group_depth defines how many stripes are accessed before advancing to the next group of components in the list of component objects for the file. The size of the components array MUST be a multiple of group_width. The mirror_cnt is used to replicate a file by replicating its component objects. If there is no mirroring, then mirror_cnt MUST be 0. If mirror_cnt is greater than zero, then the size of the Halevy, et al. Expires March 8, 2008 [Page 10] Internet-Draft pnfs objects September 2007 component array MUST be a multiple of (mirror_cnt+1). See Section 3.2 for more details. 3.2. Data Mapping Schemes This section describes the different data mapping schemes in detail. 3.2.1. Simple Striping The object layout always uses a "dense" layout as described in the pNFS document. This means that the second stripe unit of the file starts at offset 0 of the second component, rather than at offset stripe_unit bytes. After a full stripe has been written, the next stripe unit is appended to the first component object in the list without any holes in the component objects. The mapping from the logical offset within a file (L) to do the component object C and object-specific offset O is defined by the following equations: L = logical offset into the file W = total number of components S = W * stripe_unit N = L / S C = (L-(N*S)) / stripe_unit O = (N*stripe_unit)+(L%stripe_unit) In these equations, S is the number of bytes in a full stripe, and N is the stripe number. C is an index into the array of components, so it selects a particular object storage device. Both N and C count from zero. O is the offset within the object that corresponds to the file offset. Note that this computation does not accommodate the same object appearing in the component array multiple times. For example, consider an object striped over four devices, . The stripe_unit is 4096 bytes. The stripe width S is thus 4 * 4096 = 16384. Halevy, et al. Expires March 8, 2008 [Page 11] Internet-Draft pnfs objects September 2007 Offset 0: N = 0 / 16384 = 0 C = 0-0/4096 = 0 (D0) O = 0*4096 + (0%4096) = 0 Offset 4096: N = 4096 / 16384 = 0 C = (4096-(0*16384)) / 4096 = 1 (D1) O = (0*4096)+(4096%4096) = 0 Offset 9000: N = 9000 / 16384 = 0 C = (9000-(0*16384)) / 4096 = 2 (D2) O = (0*4096)+(9000%4096) = 808 Offset 132000: N = 132000 / 16384 = 8 C = (132000-(8*16384)) / 4096 = 0 O = (8*4096) + (132000%4096) = 33696 3.2.2. Nested Striping The group_width and group_depth parameters allow a nested striping pattern. If there is no nesting, then group_width and group_depth MUST be zero. Otherwise, the group_width defines the width of a data stripe, and the group_depth defines how many stripes are written before advancing to the next group of components in the list of component objects for the file. The size of the components array MUST be a multiple of group_width. The math used to map from a file offset to a component object and offset within that object is shown below. The computations map from the logical offset L to the component index C and offset relative O within that component object. L = logical offset into the file W = total number of components S = stripe_unit * group_depth * W T = stripe_unit * group_depth * group_width U = stripe_unit * group_width M = L / S G = (L - (M * S)) / T H = (L - (M * S)) % T N = H / U C = (H - (N * U)) / stripe_unit + G * group_width O = L % stripe_unit + N * stripe_unit + M * group_depth * stripe_unit In these equations, S is the number of bytes striped across all component objects before the pattern repeats. T is the number of bytes striped within a group of component objects before advancing to Halevy, et al. Expires March 8, 2008 [Page 12] Internet-Draft pnfs objects September 2007 the next group. U is the number of bytes in a stripe within a group. M is the "major" (i.e., across all components) stripe number, and N is the "minor" (i.e., across the group) stripe number. G counts the groups from the beginning of the major stripe, and H is the byte offset within the group. For example, consider an object striped over 100 devices with a group_width of 10, a group_depth of 50, and a stripe_unit of 1 MB. In this scheme, 500 MB are written to the first 10 components, and 5000 MB is written before the pattern wraps back around to the first component in the array. Offset 0: W = 100 S = 1 MB * 50 * 100 = 5000 MB T = 1 MB * 50 * 10 = 500 MB U = 1 MB * 10 = 10 MB M = 0 / 5000 MB = 0 G = (0 - (0 * 5000 MB)) / 500 MB = 0 H = (0 - (0 * 5000 MB)) % 500 MB = 0 N = 0 / 10 MB = 0 C = (0 - (0 * 10 MB)) / 1 MB + 0 * 10 = 0 O = 0 % 1 MB + 0 * 1 MB + 0 * 50 * 1 MB = 0 Offset 27 MB: M = 27 MB / 5000 MB = 0 G = (27 MB - (0 * 5000 MB)) / 500 MB = 0 H = (27 MB - (0 * 5000 MB)) % 500 MB = 27 MB N = 27 MB / 10 MB = 2 C = (27 MB - (2 * 10 MB)) / 1 MB + 0 * 10 = 7 O = 27 MB % 1 MB + 2 * 1 MB + 0 * 50 * 1 MB = 2 MB Offset 7232 MB: M = 7232 MB / 5000 MB = 1 G = (7232 MB - (1 * 5000 MB)) / 500 MB = 4 H = (7232 MB - (1 * 5000 MB)) % 500 MB = 232 MB N = 232 MB / 10 MB = 23 C = (232 MB - (23 * 10 MB)) / 1 MB + 4 * 10 = 42 O = 7232 MB % 1 MB + 23 * 1 MB + 1 * 50 * 1 MB = 73 MB 3.2.3. Mirroring The mirror_cnt is used to replicate a file by replicating its component objects. If there is no mirroring, then mirror_cnt MUST be 0. If mirror_cnt is greater than zero, then the size of the component array MUST be a multiple of (mirror_cnt+1). Thus, for a classic mirror on two objects, mirror_cnt is one. If group_width is also non-zero, then the size MUST be a multiple of group_width * Halevy, et al. Expires March 8, 2008 [Page 13] Internet-Draft pnfs objects September 2007 (mirror_cnt+1). Replicas are adjacent in the components array, and the value C produced by the above equations is not a direct index into the components array. Instead, the following equations determine the replica component index RCi, where i ranges from 0 to mirror_cnt. C = component index for striping or two-level striping i ranges from 0 to mirror_cnt, inclusive RCi = C * (mirror_cnt+1) + i 3.3. RAID Algorithms pnfs_osd_raid_algorithm4 determines the algorithm and placement of redundant data. This section defines the different RAID algorithms. 3.3.1. PNFS_OSD_RAID_0 PNFS_OSD_RAID_0 means there is no parity data, so all bytes in the component objects are data bytes located by the above equations for C and O. If a component object is unavailable, the pNFS client can choose to return NULLs for the missing data, or it can retry the READ against the pNFS server, or it can return an EIO error. 3.3.2. PNFS_OSD_RAID_4 PNFS_OSD_RAID_4 means that the last component object, or the last in each group if group_width is > zero, contains parity information computed over the rest of the stripe with an XOR operation. If a component object is unavailable, the client can read the rest of the stripe units in the damaged stripe and recompute the missing stripe unit by XORing the other stripe units in the stripe. Or the client can replay the READ against the pNFS server which will presumably perform the reconstructed read on the client's behalf. When parity is present in the file, then there is an additional computation to map from the file offset L to the offset that accounts for embedded parity, L'. First compute L', and then use L' in the above equations for C and O. L = file offset, not accounting for parity P = number of parity devices in each stripe W = group_width, if not zero, else size of component array N = L / (W-P * stripe_unit) L' = N * (W * stripe_unit) + (L % (W-P * stripe_unit)) Halevy, et al. Expires March 8, 2008 [Page 14] Internet-Draft pnfs objects September 2007 3.3.3. PNFS_OSD_RAID_5 PNFS_OSD_RAID_5 means that the position of the parity data is rotated on each stripe. In the first stripe, the last component holds the parity. In the second stripe, the next-to-last component holds the parity, and so on. In this scheme, all stripe units are rotated so that I/O is evenly spread across objects as the file is read sequentially. The rotated parity layout is illustrated here, with numbers indicating the stripe unit. 0 1 2 P 4 5 P 3 8 P 6 7 P 9 a b To compute the component object C, first compute the offset that accounts for parity L' and use that to compute C. Then rotate C to get C'. Finally, increase C' by one if the parity information comes at or before C' within that stripe. The following equations illustrate this by computing I, which is the index of the component that contains parity for a given stripe. L = file offset, not accounting for parity W = group_width, if not zero, else size of component array N = L / (W-1 * stripe_unit) (Compute L' as describe above) (Compute C based on L' as described above) C' = (C - (N%W)) % W I = W - (N%W) - 1 if (C' <= I) { C'++ } 3.3.4. PNFS_OSD_RAID_PQ PNFS_OSD_RAID_PQ is a double-parity scheme that uses the Reed-Solomon P+Q encoding scheme. In this layout, the last two component objects hold the P and Q data, respectively. P is parity computed with XOR, and Q is a more complex equation that is not described here. The equations given above for embedded parity can be used to map a file offset to the correct component object by setting the number of parity components to 2 instead of 1 for RAID4 or RAID5. Clients may simply choose to read data through the metadata server if two components are missing or damaged. Issue: This scheme also has a RAID_4 like layout where the ECC blocks are stored on the same components on every stripe and a rotated, RAID-5 like layout where the stripe units are rotated. Should we Halevy, et al. Expires March 8, 2008 [Page 15] Internet-Draft pnfs objects September 2007 make the following properties orthogonal: RAID_4 or RAID_5 (i.e., non-rotated or rotated), and then have the number of parity components and the associated algorithm be the orthogonal parameter? 3.3.5. RAID Usage and implementation notes RAID layouts with redundant data in their stripes require additional serialization of updates to ensure correct operation. Otherwise, if two clients simultaneously write to the same logical range of an object, the result could include different data in the same ranges of mirrored tuples, or corrupt parity information. It is the responsibility of the metadata server to enforce serialization requirements such as this. For example, the metadata server may do so by not granting overlapping write layouts within mirrored objects. 4. Object-Based Layout Update layoutupdate4 is used in the LAYOUTCOMMIT operation to convey updates to the layout and additional information to the metadata server. It is defined in the NFSv4.1 draft [6] as follows: struct layoutupdate4 { layouttype4 lou_type; opaque lou_body<>; }; The layoutupdate4 type is an opaque value at the generic pNFS client level. If the lou_type layout type is LAYOUT4_OSD2_OBJECTS, then the lou_body opaque value is defined by the pnfs_osd_layoutupdate4 type. 4.1. pnfs_osd_layoutupdate4 struct pnfs_osd_layoutupdate4 { pnfs_osd_deltaspaceused4 delta_space_used; pnfs_osd_ioerr4 ioerr<>; }; Object-Based pNFS clients are not allowed to modify the layout. "delta_space_used" is used to convey capacity usage information back to the metadata server and, in case OSD I/O operations failed, "ioerr" is used to report these errors to the metadata server. Halevy, et al. Expires March 8, 2008 [Page 16] Internet-Draft pnfs objects September 2007 4.1.1. pnfs_osd_deltaspaceused4 union pnfs_osd_deltaspaceused4 switch (bool valid) { case TRUE: int64_t delta; /* Bytes consumed by write activity */ case FALSE: void; }; pnfs_osd_deltaspaceused4 is used to convey space utilization information at the time of LAYOUTCOMMIT. For the file system to properly maintain capacity used information, it needs to track how much capacity was consumed by WRITE operations performed by the client. In this protocol, the OSD returns the capacity consumed by a write, which can be different than the number of bytes written because of internal overhead like block-based allocation and indirect blocks, and the client reflects this back to the pNFS server so it can accurately track quota. The pNFS server can choose to trust this information coming from the clients and therefore avoid querying the OSDs at the time of LAYOUTCOMMIT. If the client is unable to obtain this information from the OSD, it simply returns invalid delta_space_used. 4.1.2. pnfs_osd_errno4 enum pnfs_osd_errno4 { PNFS_OSD_ERR_EIO = 1, PNFS_OSD_ERR_NOT_FOUND = 2, PNFS_OSD_ERR_NO_SPACE = 3, PNFS_OSD_ERR_BAD_CRED = 4, PNFS_OSD_ERR_NO_ACCESS = 5, PNFS_OSD_ERR_UNREACHABLE = 6, PNFS_OSD_ERR_RESOURCE = 7 }; pnfs_osd_errno4 is used to represent error types when read/write errors are reported to the metadata server. The error codes serve as hints to the metadata server that may help it in diagnosing the exact reason for the error and in repairing it. o PNFS_OSD_ERR_EIO indicates the operation failed because the Object Storage Device experienced a failure trying to access the object. The most common source of these errors is media errors, but other internal errors might cause this. In this case, the metadata server should go examine the broken object more closely, hence it should be used as the default error code. Halevy, et al. Expires March 8, 2008 [Page 17] Internet-Draft pnfs objects September 2007 o PNFS_OSD_ERR_NOT_FOUND indicates the object ID specifies an object that does not exist on the Object Storage Device. o PNFS_OSD_ERR_NO_SPACE indicates the operation failed because the Object Storage Device ran out of free capacity during the operation. o PNFS_OSD_ERR_BAD_CRED indicates the security parameters are not valid. The primary cause of this is that the capability has expired, or the access policy tag (a.k.a, capability version number) has been changed to revoke capabilities. The client will need to return the layout and get a new one with fresh capabilities. o PNFS_OSD_ERR_NO_ACCESS indicates the capability does not allow the requested operation. This should not occur in normal operation because the metadata server should give out correct capabilities, or none at all. o PNFS_OSD_ERR_UNREACHABLE indicates the client did not complete the I/O operation at the Object Storage Device due to a communication failure. Whether the I/O operation was executed by the OSD or not is undetermined. o PNFS_OSD_ERR_RESOURCE indicates the client did not issue the I/O operation due to a local problem on the initiator (i.e. client) side, e.g., when running out of memory. The client MUST guarantee that the OSD command was never dispatched to the OSD. 4.1.3. pnfs_osd_ioerr4 struct pnfs_osd_ioerr4 { pnfs_osd_objid4 component; length4 comp_offset; length4 comp_length; bool iswrite; pnfs_osd_errno4 errno; }; The pnfs_osd_ioerr4 structure is used to return error indications for objects that generated errors during data transfers. These are hints to the metadata server that there are problems with that object. For each error, "component", "comp_offset", and "comp_length" represent the object and byte range within the component object in which the error occurred. "iswrite" is set to "true" if the failed OSD operation was data modifying, and "errno" represents the type of error. Halevy, et al. Expires March 8, 2008 [Page 18] Internet-Draft pnfs objects September 2007 5. Object-Based Creation Layout Hint The layouthint4 type is defined in the NFSv4.1 draft [6] as follows: struct layouthint4 { layouttype4 loh_type; opaque loh_body<>; }; The layouthint4 structure is used by the client to pass in a hint about the type of layout it would like created for a particular file. If the loh_type layout type is LAYOUT4_OSD2_OBJECTS, then the loh_body opaque value is defined by the pnfs_osd_layouthint4 type. 5.1. pnfs_osd_layouthint4 union num_comps_hint4 switch (bool valid) { case TRUE: uint32_t num_comps; case FALSE: void; }; union stripe_unit_hint4 switch (bool valid) { case TRUE: length4 stripe_unit; case FALSE: void; }; union group_width_hint4 switch (bool valid) { case TRUE: uint32_t group_width; case FALSE: void; }; union group_depth_hint4 switch (bool valid) { case TRUE: uint32_t group_depth; case FALSE: void; }; union mirror_cnt_hint4 switch (bool valid) { case TRUE: uint32_t mirror_cnt; case FALSE: Halevy, et al. Expires March 8, 2008 [Page 19] Internet-Draft pnfs objects September 2007 void; }; union raid_algorithm_hint4 switch (bool valid) { case TRUE: pnfs_osd_raid_algorithm4 raid_algorithm; case FALSE: void; }; struct pnfs_osd_layouthint4 { num_comps_hint4 num_comps_hint; stripe_unit_hint4 stripe_unit_hint; group_width_hint4 group_width_hint; group_depth_hint4 group_depth_hint; mirror_cnt_hint4 mirror_cnt_hint; raid_algorithm_hint4 raid_algorithm_hint; }; This type conveys hints for the desired data map. All parameters are optional so the client can give values for only the parameters it cares about, e.g. it can provide a hint for the desired number of mirrored components, regardless of the the raid algorithm selected for the file. The server should make an attempt to honor the hints but it can ignore any or all of them at its own discretion and without failing the respective create operation. The num_comps hint can be used to limit the total number of component objects comprising the file. All other hints correspond directly to the different fields of pnfs_osd_data_map4. 6. Layout Segments The pnfs layout operations operate on logical byte ranges. There is no requirement in the protocol for any relationship between byte ranges used in LAYOUTGET to acquire layouts and byte ranges used in CB_LAYOUTRECALL, LAYOUTCOMMIT, or LAYOUTRETURN. However, using OSD capabilities poses limitations on these operations since the capabilities associated with layout segments cannot be merged or split. The following guidelines should be followed for proper operation of object-based layouts. 6.1. CB_LAYOUTRECALL and LAYOUTRETURN In general, the object-based layout driver should keep track of each layout segment it got, keeping record of the segment's iomode, offset, and length. The server should allow the client to get Halevy, et al. Expires March 8, 2008 [Page 20] Internet-Draft pnfs objects September 2007 multiple overlapping layout segments but is free to recall the layout to prevent overlap. In response to CB_LAYOUTRECALL, the client should return all layout segments matching the given iomode and overlapping with the recalled range. When returning the layouts for this byte range with LAYOUTRETURN the client MUST NOT return a sub-range of a layout segment it has; each LAYOUTRETURN sent MUST completely cover at least one outstanding layout segment. The server, in turn, should release any segment that exactly matches the clientid, iomode, and byte range given in LAYOUTRETURN. If no exact match is found then the server should release all layout segments matching the clientid and iomode and that are fully contained in the returned byte range. If none are found and the byte range is a subset of an outstanding layout segment with for the same clientid and iomode, then the client can be considered malfunctioning and the server SHOULD recall all layouts from this client to reset its state. If this behavior repeats the server SHOULD deny all LAYOUTGETs from this client. 6.2. LAYOUTCOMMIT LAYOUTCOMMIT is only used by object-based pNFS to convey modified attributes hints and/or to report I/O errors to the MDS. Therefore, the offset and length in LAYOUTCOMMIT4args are reserved for future use and should be set to 0. However, component byte ranges in the optional pnfs_osd_ioerr4 structure are used for recovering the object and MUST be set by the client to cover all failed I/O operations to the component. 7. Recalling Layouts The object-based metadata server should recall outstanding layouts in the following cases: o When the file's security policy changes, i.e. ACLs or permission mode bits are set. o When the file's aggregation map changes, rendering outstanding layouts invalid. o When there are sharing conflicts. For example, the server will issue stripe aligned layout segments for RAID-5 objects. To prevent corruption of the file's parity, Multiple clients must not hold valid write layouts for the same stripes. An outstanding RW layout should be recalled when a conflicting LAYOUTGET is received Halevy, et al. Expires March 8, 2008 [Page 21] Internet-Draft pnfs objects September 2007 from a different client for LAYOUTIOMODE4_RW and for a byte-range overlapping with the outstanding layout segment. 7.1. CB_RECALL_ANY The metadata server can use the CB_RECALL_ANY callback operation to notify the client to return some or all of its layouts. The NFSv4.1 draft [6] defines the following types: const RCA4_TYPE_MASK_OBJ_LAYOUT_MIN = 8; const RCA4_TYPE_MASK_OBJ_LAYOUT_MAX = 11; struct CB_RECALL_ANY4args { uint32_t craa_objects_to_keep; bitmap4 craa_type_mask; }; Typically, CB_RECALL_ANY will be used to recall client state when the server needs to reclaim resources. The craa_type_mask bitmap specifies the type of resources that are recalled and the craa_objects_to_keep value specifies how many of the recalled objects the client is allowed to keep. The object-based layout type mask flags are defined as follows. They represent the iomode of the recalled layouts. In response, the client SHOULD return layouts of the recalled iomode that it needs the least, keeping at most craa_objects_to_keep object-based layouts. const PNFS_OSD_RCA4_TYPE_MASK_READ = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN; const PNFS_OSD_RCA4_TYPE_MASK_RW = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN+1; const PNFS_OSD_RCA4_TYPE_MASK_ANY = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN+2; The PNFS_OSD_RCA4_TYPE_MASK_READ flag notifies the client to return layouts of iomode LAYOUTIOMODE4_READ. Similarly, the PNFS_OSD_RCA4_TYPE_MASK_RW flag notifies the client to return layouts of iomode LAYOUTIOMODE4_RW. The PNFS_OSD_RCA4_TYPE_MASK_ANY flag notifies the client to return layouts of either iomode. 8. Client Fencing In cases where clients are uncommunicative and their lease has expired or when clients fail to return recalled layouts in a timely manner the server MAY revoke client layouts and/or device address mappings and reassign these resources to other clients. To avoid data corruption, the metadata server MUST fence off the revoked clients from the respective objects as described in Section 9.4. Halevy, et al. Expires March 8, 2008 [Page 22] Internet-Draft pnfs objects September 2007 9. Security Considerations The pNFS extension partitions the NFSv4 file system protocol into two parts, the control path and the data path (storage protocol). The control path contains all the new operations described by this extension; all existing NFSv4 security mechanisms and features apply to the control path. The combination of components in a pNFS system is required to preserve the security properties of NFSv4 with respect to an entity accessing data via a client, including security countermeasures to defend against threats that NFSv4 provides defenses for in environments where these threats are considered significant. The metadata server enforces the file access-control policy at LAYOUTGET time. The client should use suitable authorization credentials for getting the layout for the requested iomode (READ or RW) and the server verifies the permissions and ACL for these credentials, possibly returning NFS4ERR_ACCESS if the client is not allowed the requested iomode. If the LAYOUTGET operation succeeds the client receives, as part of the layout, a set of object capabilities allowing it I/O access to the specified objects corresponding to the requested iomode. When the client acts on I/O operations on behalf of its local users it MUST authenticate and authorize the user by issuing respective OPEN and ACCESS calls to the metadata server, similarly to having NFSv4 data delegations. If access is allowed the client uses the corresponding (READ or RW) capabilities to perform the I/O operations at the object-storage devices. When the metadata server receives a request to change file's permissions or ACL it SHOULD recall all layouts for that file and it MUST change the capability version attribute on all objects comprising the file to implicitly invalidate any outstanding capabilities before committing to the new permissions and ACL. Doing this will ensure that clients re-authorize their layouts according to the modified permissions and ACL by requesting new layouts. Recalling the layouts in this case is courtesy of the server intended to prevent clients from getting an error on I/Os done after the capability version changed. The object storage protocol MUST implement the security aspects described in version 1 of the T10 OSD protocol definition [2]. The standard defines four security methods: NOSEC, CAPKEY, CMDRSP, and ALLDATA. To provide minimum level of security allowing verification and enforcement of the server access control policy using the layout security credentials, the NOSEC security method MUST NOT be used for I/O operation. It MAY only be used to get the System ID attribute when the metadata server provided only the OSD name with the device address. The remainder of this section gives an overview of the security mechanism described in that standard. The goal is to give Halevy, et al. Expires March 8, 2008 [Page 23] Internet-Draft pnfs objects September 2007 the reader a basic understanding of the object security model. Any discrepancies between this text and the actual standard are obviously to be resolved in favor of the OSD standard. 9.1. OSD Security Data Types There are three main data types associated with object security: a capability, a credential, and security parameters. The capability is a set of fields that specifies an object and what operations can be performed on it. A credential is a signed capability. Only a security manager that knows the secret device keys can correctly sign a capability to form a valid credential. In pNFS, the file server acts as the security manager and returns signed capabilities (i.e., credentials) to the pNFS client. The security parameters are values computed by the issuer of OSD commands (i.e., the client) that prove they hold valid credentials. The client uses the credential as a signing key to sign the requests it makes to OSD, and puts the resulting signatures into the security_parameters field of the OSD command. The object storage device uses the secret keys it shares with the security manager to validate the signature values in the security parameters. The security types are opaque to the generic layers of the pNFS client. The credential contents are defined as opaque within the pnfs_osd_object_cred4 type. Instead of repeating the definitions here, the reader is referred to section 4.9.2.2 of the OSD standard. 9.2. The OSD Security Protocol The object storage protocol relies on a cryptographically secure capability to control accesses at the object storage devices. Capabilities are generated by the metadata server, returned to the client, and used by the client as described below to authenticate their requests to the Object Storage Device (OSD). Capabilities therefore achieve the required access and open mode checking. They allow the file server to define and check a policy (e.g., open mode) and the OSD to enforce that policy without knowing the details (e.g., user IDs and ACLs). Since capabilities are tied to layouts, and since they are used to enforce access control, when the file ACL or mode changes the outstanding capabilities MUST be revoked to enforce the new access permissions. The server SHOULD recall layouts to allow clients to gracefully return their capabilities before the access permissions change. Each capability is specific to a particular object, an operation on that object, a byte range w/in the object (in OSDv2), and has an Halevy, et al. Expires March 8, 2008 [Page 24] Internet-Draft pnfs objects September 2007 explicit expiration time. The capabilities are signed with a secret key that is shared by the object storage devices (OSD) and the metadata managers. Clients do not have device keys so they are unable to forge the signatures in the security parameters. The combination of a capability, the OSD system id, and a signature is called a "credential" in the OSD specification. The details of the security and privacy model for Object Storage are defined in the T10 OSD standard. The following sketch of the algorithm should help the reader understand the basic model. LAYOUTGET returns a CapKey and a Cap which, together with the OSD SystemID, are also called a credential. It is a capability and a signature over that capability and the SystemID. The OSD Standard refers to the CapKey as the "Credential integrity check value" and to the ReqMAC as the "Request integrity check value". CapKey = MAC(Cap, SystemID) Credential = {Cap, SystemID, CapKey} The client uses CapKey to sign all the requests it issues for that object using the respective Cap. In other words, the Cap appears in the request to the storage device, and that request is signed with the CapKey as follows: ReqMAC = MAC(Req, ReqNonce) Request = {Cap, Req, ReqNonce, ReqMAC} The following is sent to the OSD: {Cap, Req, ReqNonce, ReqMAC}. The OSD uses the SecretKey it shares with the metadata server to compare the ReqMAC the client sent with a locally computed value: LocalCapKey = MAC(Cap, SystemID) LocalReqMAC = MAC(Req, ReqNonce) and if they match the OSD assumes that the capabilities came from an authentic metadata server and allows access to the object, as allowed by the Cap. 9.3. Protocol Privacy Requirements Note that if the server LAYOUTGET reply, holding CapKey and Cap, is snooped by another client, it can be used to generate valid OSD requests (within the Cap access restrictions). To provide the required privacy requirements for the capability key returned by LAYOUTGET, the GSS-API can be used, e.g. by using the RPCSEC_GSS privacy method to send the LAYOUTGET operation or by using Halevy, et al. Expires March 8, 2008 [Page 25] Internet-Draft pnfs objects September 2007 the SSV key to encrypt the capability_key using the GSS_Wrap() function. Two general ways to provide privacy in the absence of GSS- API that are independent of NFSv4 are either an isolated network such as a VLAN or a secure channel provided by IPsec [8]. 9.4. Revoking Capabilities At any time, the metadata server may invalidate all outstanding capabilities on an object by changing its POLICY ACCESS TAG attribute. The value of the POLICY ACCESS TAG is part of a capability, and it must match the state of the object attribute. If they do not match, the OSD rejects accesses to the object with the sense key set to ILLEGAL REQUEST and an additional sense code set to INVALID FIELD IN CDB. When a client attempts to use a capability and is rejected this way, it should issue a LAYOUTCOMMIT for the object and specify PNFS_OSD_BAD_CRED in the ioerr parameter. The client may elect to issue a compound LAYOUTRETURN/LAYOUTGET (or LAYOUTCOMMIT/ LAYOUTRETURN/LAYOUTGET) to attempt to fetch a refreshed set of capabilities. The metadata server may elect to change the access policy tag on an object at any time, for any reason (with the understanding that there is likely an associated performance penalty, especially if there are outstanding layouts for this object). The metadata server MUST revoke outstanding capabilities when any one of the following occurs: o the permissions on the object change, o a conflicting mandatory byte-range lock is granted, or o a layout is revoked and reassigned to another client A pNFS client will typically hold one layout for each byte range for either READ or READ/WRITE. The client's credentials are checked by the metadata server at LAYOUTGET time and it is the client's responsibility to enforce access control among multiple users accessing the same file. It is neither required nor expected that the pNFS client will obtain a separate layout for each user accessing a shared object. The client SHOULD use OPEN and ACCESS calls to check user permissions when performing I/O so that the server's access control policies are correctly enforced. The result of the ACCESS operation may be cached while the client holds a valid layout as the server is expected to recall layouts when the file's access permissions or ACL change. Halevy, et al. Expires March 8, 2008 [Page 26] Internet-Draft pnfs objects September 2007 10. IANA Considerations As described in the NFSv4.1 draft [6], new layout type numbers will be requested from IANA. This document defines the protocol associated with the existing layout type number, LAYOUT4_OSD2_OBJECTS, and it requires no further actions for IANA. 11. References 11.1. Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [2] Weber, R., "SCSI Object-Based Storage Device Commands", July 2004, . [3] Linn, J., "Generic Security Service Application Program Interface Version 2, Update 1", RFC 2743, January 2000. [4] Eisler, M., "XDR: External Data Representation Standard", STD 67, RFC 4506, May 2006. [5] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M., and D. Noveck, "Network File System (NFS) version 4 Protocol", RFC 3530, April 2003. 11.2. Informative References [6] Shepler, S., Eisler, M., and D. Noveck, "NFSv4 Minor Version 1", March 2007, . [7] Weber, R., "SCSI Object-Based Storage Device Commands -2 (OSD-2)", January 2007, . [8] Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, December 2005. Appendix A. Acknowledgments Todd Pisek was a co-editor of the initial drafts for this document. Halevy, et al. Expires March 8, 2008 [Page 27] Internet-Draft pnfs objects September 2007 Authors' Addresses Benny Halevy Panasas, Inc. 1501 Reedsdale St. Suite 400 Pittsburgh, PA 15233 USA Phone: +1-412-323-3500 Email: bhalevy@panasas.com URI: http://www.panasas.com/ Brent Welch Panasas, Inc. 6520 Kaiser Drive Fremont, CA 95444 USA Phone: +1-650-608-7770 Email: welch@panasas.com URI: http://www.panasas.com/ Jim Zelenka Panasas, Inc. 1501 Reedsdale St. Suite 400 Pittsburgh, PA 15233 USA Phone: +1-412-323-3500 Email: jimz@panasas.com URI: http://www.panasas.com/ Halevy, et al. Expires March 8, 2008 [Page 28] Internet-Draft pnfs objects September 2007 Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Halevy, et al. Expires March 8, 2008 [Page 29] --------------010303050605070809060207 Content-Type: text/xml; name="draft-ietf-nfsv4-pnfs-obj-04.xml" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="draft-ietf-nfsv4-pnfs-obj-04.xml" Object-based pNFS Operations Panasas, Inc.

1501 Reedsdale St. Suite 400 Pittsburgh PA 15233 USA +1-412-323-3500 bhalevy@panasas.com http://www.panasas.com/

Panasas, Inc.

6520 Kaiser Drive Fremont CA 95444 USA +1-650-608-7770 welch@panasas.com http://www.panasas.com/

Panasas, Inc.

1501 Reedsdale St. Suite 400 Pittsburgh PA 15233 USA +1-412-323-3500 jimz@panasas.com http://www.panasas.com/

Transport NFSv4 This Internet-Draft provides a description of the object-based pNFS extension for NFSv4. This is a companion to the main pnfs specification in the NFSv4 Minor Version 1 Internet Draft, which is currently draft-ietf-nfsv4-minorversion1-13.txt. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

In pNFS, the file server returns typed layout structures that describe where file data is located. There are different layouts for different storage systems and methods of arranging data on storage devices. This document describes the layouts used with object-based storage devices (OSD) that are accessed according to the iSCSI/OSD storage protocol standard (SNIA T10/1355-D). An "object" is a container for data and attributes, and files are stored in one or more objects. The OSD protocol specifies several operations on objects, including READ, WRITE, FLUSH, GET ATTRIBUTES, SET ATTRIBUTES, CREATE and DELETE. However, using the object-based layout the client only uses the READ, WRITE, GET ATTRIBUTES and FLUSH commands. The other commands are only used by the pNFS server. An object-based layout for pNFS includes object identifiers, capabilities that allow clients to READ or WRITE those objects, and various parameters that control how file data is striped across their component objects. The OSD protocol has a capability-based security scheme that allows the pNFS server to control what operations and what objects can be used by clients. This scheme is described in more detail in the Security Considerations section.

Data operations to an OSD require the client to know the "address" of each OSD's root object. The root object is synonymous with SCSI logical unit. The client specifies SCSI logical units to its SCSI stack using a representation local to the client. Because these representations are local, GETDEVICEINFO must return information that can be used by the client to select the correct local representation. In the block world, a set offset (logical block number or track/sector) contains a disk label. This label identifies the disk uniquely. In contrast, an OSD has a standard set of attributes on its root object. For device identification purposes the OSD System ID (root information attribute number 3) and/or OSD Name (root information attribute number 9) are used as the label. These appear in the pnfs_osd_deviceaddr4 type below under the "systemid" and "osdname" fields. In some situations, SCSI target discovery may need to be driven based on information contained in the GETDEVICEINFO response. One example of this is iSCSI targets that are not known to the client until a layout has been requested. Eventually iSCSI will adopt ANSI T10 SAM-3, at which time the World Wide Name (WWN aka, EUI-64/EUI-128) naming conventions can be specified. In addition, Fibre Channel (FC) SCSI targets have a unique WWN. Although these FC targets have already been discovered, some implementations may want to specify the WWN in addition to the label. This information appears as the "target" and "lun" fields in the pnfs_osd_deviceaddr4 type described below. The systemid is used by the client, along with the object credential to sign each request with the request integrity check value. This method protects the client from unintentionally accessing a device if the device address mapping was changed (or revoked). The server computes the capability_key using its own view of the systemid associated with the respective deviceid present in the credential. If the client's view of the deviceid mapping is stale, the client will use the wrong systemid (which must be system-wide unique) and the I/O request to the OSD will fail to pass the integrity check verification. To recover from this condition the client should report the error via LAYOUTCOMMIT, return the layout using LAYOUTRETURN, and invalidate all the device address mappings associated with this layout. The client can then ask for a new layout if it wishes using LAYOUTGET and resolve the referenced deviceids using GETDEVICEINFO or GETDEVICELIST. The server MUST provide either the systemid, the OSD name, or both. When the OSD name is present the client SHOULD get the root information attributes whenever it establishes communication with the OSD and verify that the OSD name it got from the OSD matches the one sent by the metadata server. If the systemid was not given by the server it MUST be taken from the OSD-provided attribute; note that in this case the OSD GET ATTRIBUTES operation must be performed with the NOSEC security method.

The following enum specifies the manner in which a scsi target can be specified. The target can be specified as a network address, as an Internet Qualified Name (IQN), or by the World-Wide Name (WWN) of the target.

enum pnfs_obj_addr_type4 { OBJ_TARGET_NETADDR = 1, OBJ_TARGET_IQN = 2, OBJ_TARGET_WWN = 3 };

The specification for an object device address is as follows:

struct pnfs_osd_deviceaddr4 { union target switch (pnfs_osd_addr_type4 type) { case OBJ_TARGET_NETADDR: pnfs_netaddr4 netaddr; case OBJ_TARGET_IQN: string iqn<>; case OBJ_TARGET_WWN: string wwn<>; default: void; }; uint64_t lun; opaque systemid<>; opaque osdname<>; };

The layout4 type is defined in the NFSv4.1 draft as follows:

enum layouttype4 { LAYOUT4_NFSV4_1_FILES = 1, LAYOUT4_OSD2_OBJECTS = 2, LAYOUT4_BLOCK_VOLUME = 3 }; struct layout_content4 { layouttype4 loc_type; opaque loc_body<>; }; struct layout4 { offset4 lo_offset; length4 lo_length; layoutiomode4 lo_iomode; layout_content4 lo_content; }; This document defines structure associated with the layouttype4 value, LAYOUT4_OSD2_OBJECTS. The NFSv4.1 draft specifies the loc_body structure as an XDR type "opaque". The opaque layout is uninterpreted by the generic pNFS client layers, but obviously must be interpreted by the object-storage layout driver. This document defines the structure of this opaque value, pnfs_osd_layout4.

struct pnfs_osd_layout4 { pnfs_osd_data_map4 map; pnfs_osd_object_cred4 components<>; }; The pnfs_osd_layout4 structure specifies a layout over a set of component objects. The components field is an array of object identifiers and security credentials that grant access to each object. The organization of the data is defined by the pnfs_osd_data_map4 type that specifies how the file's data is mapped onto the component objects (i.e., the striping pattern). The data placement algorithm that maps file data onto component objects assume that each component object occurs exactly once in the array of components. Therefore, component objects MUST appear in the component array only once. Note that the layout depends on the file size, which the client learns from the generic return parameters of LAYOUTGET, by doing GETATTR commands to the metadata server. The client uses the file size to decide if it should fill holes with zeros, or return a short read. Striping patterns can cause cases where component objects are shorter than other components because a hole happens to correspond to the last part of the component object.

An object is identified by a number, somewhat like an inode number. The object storage model has a two level scheme, where the objects within an object storage device are grouped into partitions.

struct pnfs_osd_objid4 { deviceid4 device_id; uint64_t partition_id; uint64_t object_id; }; The pnfs_osd_objid4 type is used to identify an object within a partition on a specified object storage device. "device_id" selects the object storage device from the set of available storage devices. The device is identified with the deviceid4 type, which is an index into addressing information about that device returned by the GETDEVICELIST and GETDEVICEINFO pnfs operations. Within an OSD, a partition is identified with a 64-bit number, "partition_id". Within a partition, an object is identified with a 64-bit number, "object_id". Creation and management of partitions is outside the scope of this standard, and is a facility provided by the object storage file system.

enum pnfs_osd_version4 { PNFS_OSD_MISSING = 0, PNFS_OSD_VERSION_1 = 1, PNFS_OSD_VERSION_2 = 2 }; The osd_version is used to indicate the OSD protocol version or whether an object is missing (i.e., unavailable). Some layout schemes encode redundant information and can compensate for missing components, but the data placement algorithm needs to know what parts are missing. At this time the OSD standard is at version 1.0, and we anticipate a version 2.0 of the standard ((SNIA T10/1729-D)). The second generation OSD protocol has additional proposed features to support more robust error recovery, snapshots, and byte-range capabilities. Therefore, the OSD version is explicitly called out in the information returned in the layout. (This information can also be deduced by looking inside the capability type at the format field, which is the first byte. The format value is 0x1 for an OSD v1 capability. However, it seems most robust to call out the version explicitly.)

enum pnfs_osd_cap_key_sec4 { PNFS_OSD_CAP_KEY_SEC_NONE = 0, PNFS_OSD_CAP_KEY_SEC_SSV = 1, }; struct pnfs_osd_object_cred4 { pnfs_osd_objid4 object_id; pnfs_osd_version4 osd_version; pnfs_osd_cap_key_sec4 cap_key_sec; opaque capability_key<>; opaque capability<>; }; The pnfs_osd_object_cred4 structure is used to identify each component comprising the file. The object_id identifies the component object, the osd_version represents the osd protocol version, or whether that component is unavailable, and the capability and capability key, along with the systemid from the pnfs_osd_deviceaddr, provide the OSD security credentials needed to access that object. The cap_key_sec value denotes the method used to secure the capability_key (see for more details). To comply with the OSD security requirements the capability key SHOULD be transferred securely to prevent eavesdropping (see ). Therefore, a client SHOULD either issue the LAYOUTGET operation via RPCSEC_GSS with the privacy service or to previously establish an SSV for the sessions via the NFSv4.1 SET_SSV operation. The pnfs_osd_cap_key_sec4 type is used to identify the method used by the server to secure the capability key. PNFS_OSD_CAP_KEY_SEC_NONE denotes that the capability_key is not encrypted in which case the client SHOULD issue the LAYOUTGET operation with RPCSEC_GSS with the privacy service or the NFSv4.1 transport should be secured by using methods that are external to NFSv4.1 like the use of IPSEC for transporting the NFSV4.1 protocol. PNFS_OSD_CAP_KEY_SEC_SSV denotes that the capability_key contents are encrypted using the SSV GSS context and the capability key as inputs to the GSS_Wrap() function (see ) with the conf_req_flag set to TRUE. The client MUST use the secret SSV key as part of the client's GSS context to decrypt the capability key using the value of the capability_key field as the input_message to the GSS_unwrap() function. Note that to prevent eavesdropping of the SSV key the client SHOULD issue SET_SSV via RPCSEC_GSS with the privacy service. The actual method chosen depends on whether the client established a SSV key with the server and whether it issued the LAYOUTGET operation with the RPCSEC_GSS privacy method. Naturally, if the client did not establish a SSV key via SET_SSV the server MUST use the PNFS_OSD_CAP_KEY_SEC_NONE method. Otherwise, if the LAYOUTGET operation was not issued with the RPCSEC_GSS privacy method the server SHOULD secure the capability_key with the PNFS_OSD_CAP_KEY_SEC_SSV method. The server MAY use the PNFS_OSD_CAP_KEY_SEC_SSV method also when the LAYOUTGET operation was issued with the RPCSEC_GSS privacy method.

enum pnfs_osd_raid_algorithm4 { PNFS_OSD_RAID_0 = 1, PNFS_OSD_RAID_4 = 2, PNFS_OSD_RAID_5 = 3, PNFS_OSD_RAID_PQ = 4 /* Reed-Solomon P+Q */ }; pnfs_osd_raid_algorithm4 represents the data redundancy algorithm used to protect the file's contents. See for more details.

struct pnfs_osd_data_map4 { length4 stripe_unit; uint32_t group_width; uint32_t group_depth; uint32_t mirror_cnt; pnfs_osd_raid_algorithm4 raid_algorithm; }; The pnfs_osd_data_map4 structure parameterizes the algorithm that maps a file's contents over the component objects. Instead of limiting the system to simple striping scheme where loss of a single component object results in data loss, the map parameters support mirroring and more complicated schemes that protect against loss of a component object. The stripe_unit is the number of bytes placed on one component before advancing to the next one in the list of components. The number of bytes in a full stripe is stripe_unit times the number of components. In some raid schemes, a stripe includes redundant information (i.e., parity) that lets the system recover from loss or damage to a component object. The group_width and group_depth parameters allow a nested striping pattern. If there is no nesting, then group_width and group_depth MUST be zero. Otherwise, the group_width defines the width of a data stripe, and the group_depth defines how many stripes are accessed before advancing to the next group of components in the list of component objects for the file. The size of the components array MUST be a multiple of group_width. The mirror_cnt is used to replicate a file by replicating its component objects. If there is no mirroring, then mirror_cnt MUST be 0. If mirror_cnt is greater than zero, then the size of the component array MUST be a multiple of (mirror_cnt+1). See for more details.

This section describes the different data mapping schemes in detail.

The object layout always uses a "dense" layout as described in the pNFS document. This means that the second stripe unit of the file starts at offset 0 of the second component, rather than at offset stripe_unit bytes. After a full stripe has been written, the next stripe unit is appended to the first component object in the list without any holes in the component objects. The mapping from the logical offset within a file (L) to do the component object C and object-specific offset O is defined by the following equations:

L = logical offset into the file W = total number of components S = W * stripe_unit N = L / S C = (L-(N*S)) / stripe_unit O = (N*stripe_unit)+(L%stripe_unit) In these equations, S is the number of bytes in a full stripe, and N is the stripe number. C is an index into the array of components, so it selects a particular object storage device. Both N and C count from zero. O is the offset within the object that corresponds to the file offset. Note that this computation does not accommodate the same object appearing in the component array multiple times. For example, consider an object striped over four devices, <D0 D1 D2 D3>. The stripe_unit is 4096 bytes. The stripe width S is thus 4 * 4096 = 16384.

Offset 0: N = 0 / 16384 = 0 C = 0-0/4096 = 0 (D0) O = 0*4096 + (0%4096) = 0 Offset 4096: N = 4096 / 16384 = 0 C = (4096-(0*16384)) / 4096 = 1 (D1) O = (0*4096)+(4096%4096) = 0 Offset 9000: N = 9000 / 16384 = 0 C = (9000-(0*16384)) / 4096 = 2 (D2) O = (0*4096)+(9000%4096) = 808 Offset 132000: N = 132000 / 16384 = 8 C = (132000-(8*16384)) / 4096 = 0 O = (8*4096) + (132000%4096) = 33696

The group_width and group_depth parameters allow a nested striping pattern. If there is no nesting, then group_width and group_depth MUST be zero. Otherwise, the group_width defines the width of a data stripe, and the group_depth defines how many stripes are written before advancing to the next group of components in the list of component objects for the file. The size of the components array MUST be a multiple of group_width. The math used to map from a file offset to a component object and offset within that object is shown below. The computations map from the logical offset L to the component index C and offset relative O within that component object.

L = logical offset into the file W = total number of components S = stripe_unit * group_depth * W T = stripe_unit * group_depth * group_width U = stripe_unit * group_width M = L / S G = (L - (M * S)) / T H = (L - (M * S)) % T N = H / U C = (H - (N * U)) / stripe_unit + G * group_width O = L % stripe_unit + N * stripe_unit + M * group_depth * stripe_unit In these equations, S is the number of bytes striped across all component objects before the pattern repeats. T is the number of bytes striped within a group of component objects before advancing to the next group. U is the number of bytes in a stripe within a group. M is the "major" (i.e., across all components) stripe number, and N is the "minor" (i.e., across the group) stripe number. G counts the groups from the beginning of the major stripe, and H is the byte offset within the group. For example, consider an object striped over 100 devices with a group_width of 10, a group_depth of 50, and a stripe_unit of 1 MB. In this scheme, 500 MB are written to the first 10 components, and 5000 MB is written before the pattern wraps back around to the first component in the array.

Offset 0: W = 100 S = 1 MB * 50 * 100 = 5000 MB T = 1 MB * 50 * 10 = 500 MB U = 1 MB * 10 = 10 MB M = 0 / 5000 MB = 0 G = (0 - (0 * 5000 MB)) / 500 MB = 0 H = (0 - (0 * 5000 MB)) % 500 MB = 0 N = 0 / 10 MB = 0 C = (0 - (0 * 10 MB)) / 1 MB + 0 * 10 = 0 O = 0 % 1 MB + 0 * 1 MB + 0 * 50 * 1 MB = 0 Offset 27 MB: M = 27 MB / 5000 MB = 0 G = (27 MB - (0 * 5000 MB)) / 500 MB = 0 H = (27 MB - (0 * 5000 MB)) % 500 MB = 27 MB N = 27 MB / 10 MB = 2 C = (27 MB - (2 * 10 MB)) / 1 MB + 0 * 10 = 7 O = 27 MB % 1 MB + 2 * 1 MB + 0 * 50 * 1 MB = 2 MB Offset 7232 MB: M = 7232 MB / 5000 MB = 1 G = (7232 MB - (1 * 5000 MB)) / 500 MB = 4 H = (7232 MB - (1 * 5000 MB)) % 500 MB = 232 MB N = 232 MB / 10 MB = 23 C = (232 MB - (23 * 10 MB)) / 1 MB + 4 * 10 = 42 O = 7232 MB % 1 MB + 23 * 1 MB + 1 * 50 * 1 MB = 73 MB

The mirror_cnt is used to replicate a file by replicating its component objects. If there is no mirroring, then mirror_cnt MUST be 0. If mirror_cnt is greater than zero, then the size of the component array MUST be a multiple of (mirror_cnt+1). Thus, for a classic mirror on two objects, mirror_cnt is one. If group_width is also non-zero, then the size MUST be a multiple of group_width * (mirror_cnt+1). Replicas are adjacent in the components array, and the value C produced by the above equations is not a direct index into the components array. Instead, the following equations determine the replica component index RCi, where i ranges from 0 to mirror_cnt.

C = component index for striping or two-level striping i ranges from 0 to mirror_cnt, inclusive RCi = C * (mirror_cnt+1) + i

pnfs_osd_raid_algorithm4 determines the algorithm and placement of redundant data. This section defines the different RAID algorithms.

PNFS_OSD_RAID_0 means there is no parity data, so all bytes in the component objects are data bytes located by the above equations for C and O. If a component object is unavailable, the pNFS client can choose to return NULLs for the missing data, or it can retry the READ against the pNFS server, or it can return an EIO error.

PNFS_OSD_RAID_4 means that the last component object, or the last in each group if group_width is > zero, contains parity information computed over the rest of the stripe with an XOR operation. If a component object is unavailable, the client can read the rest of the stripe units in the damaged stripe and recompute the missing stripe unit by XORing the other stripe units in the stripe. Or the client can replay the READ against the pNFS server which will presumably perform the reconstructed read on the client's behalf. When parity is present in the file, then there is an additional computation to map from the file offset L to the offset that accounts for embedded parity, L'. First compute L', and then use L' in the above equations for C and O.

L = file offset, not accounting for parity P = number of parity devices in each stripe W = group_width, if not zero, else size of component array N = L / (W-P * stripe_unit) L' = N * (W * stripe_unit) + (L % (W-P * stripe_unit))

PNFS_OSD_RAID_5 means that the position of the parity data is rotated on each stripe. In the first stripe, the last component holds the parity. In the second stripe, the next-to-last component holds the parity, and so on. In this scheme, all stripe units are rotated so that I/O is evenly spread across objects as the file is read sequentially. The rotated parity layout is illustrated here, with numbers indicating the stripe unit.

0 1 2 P 4 5 P 3 8 P 6 7 P 9 a b To compute the component object C, first compute the offset that accounts for parity L' and use that to compute C. Then rotate C to get C'. Finally, increase C' by one if the parity information comes at or before C' within that stripe. The following equations illustrate this by computing I, which is the index of the component that contains parity for a given stripe.

PNFS_OSD_RAID_PQ is a double-parity scheme that uses the Reed-Solomon P+Q encoding scheme. In this layout, the last two component objects hold the P and Q data, respectively. P is parity computed with XOR, and Q is a more complex equation that is not described here. The equations given above for embedded parity can be used to map a file offset to the correct component object by setting the number of parity components to 2 instead of 1 for RAID4 or RAID5. Clients may simply choose to read data through the metadata server if two components are missing or damaged. Issue: This scheme also has a RAID_4 like layout where the ECC blocks are stored on the same components on every stripe and a rotated, RAID-5 like layout where the stripe units are rotated. Should we make the following properties orthogonal: RAID_4 or RAID_5 (i.e., non-rotated or rotated), and then have the number of parity components and the associated algorithm be the orthogonal parameter?

RAID layouts with redundant data in their stripes require additional serialization of updates to ensure correct operation. Otherwise, if two clients simultaneously write to the same logical range of an object, the result could include different data in the same ranges of mirrored tuples, or corrupt parity information. It is the responsibility of the metadata server to enforce serialization requirements such as this. For example, the metadata server may do so by not granting overlapping write layouts within mirrored objects.

layoutupdate4 is used in the LAYOUTCOMMIT operation to convey updates to the layout and additional information to the metadata server. It is defined in the NFSv4.1 draft as follows:

struct layoutupdate4 { layouttype4 lou_type; opaque lou_body<>; }; The layoutupdate4 type is an opaque value at the generic pNFS client level. If the lou_type layout type is LAYOUT4_OSD2_OBJECTS, then the lou_body opaque value is defined by the pnfs_osd_layoutupdate4 type.

struct pnfs_osd_layoutupdate4 { pnfs_osd_deltaspaceused4 delta_space_used; pnfs_osd_ioerr4 ioerr<>; }; Object-Based pNFS clients are not allowed to modify the layout. "delta_space_used" is used to convey capacity usage information back to the metadata server and, in case OSD I/O operations failed, "ioerr" is used to report these errors to the metadata server.

union pnfs_osd_deltaspaceused4 switch (bool valid) { case TRUE: int64_t delta; /* Bytes consumed by write activity */ case FALSE: void; }; pnfs_osd_deltaspaceused4 is used to convey space utilization information at the time of LAYOUTCOMMIT. For the file system to properly maintain capacity used information, it needs to track how much capacity was consumed by WRITE operations performed by the client. In this protocol, the OSD returns the capacity consumed by a write, which can be different than the number of bytes written because of internal overhead like block-based allocation and indirect blocks, and the client reflects this back to the pNFS server so it can accurately track quota. The pNFS server can choose to trust this information coming from the clients and therefore avoid querying the OSDs at the time of LAYOUTCOMMIT. If the client is unable to obtain this information from the OSD, it simply returns invalid delta_space_used.

enum pnfs_osd_errno4 { PNFS_OSD_ERR_EIO = 1, PNFS_OSD_ERR_NOT_FOUND = 2, PNFS_OSD_ERR_NO_SPACE = 3, PNFS_OSD_ERR_BAD_CRED = 4, PNFS_OSD_ERR_NO_ACCESS = 5, PNFS_OSD_ERR_UNREACHABLE = 6, PNFS_OSD_ERR_RESOURCE = 7 }; pnfs_osd_errno4 is used to represent error types when read/write errors are reported to the metadata server. The error codes serve as hints to the metadata server that may help it in diagnosing the exact reason for the error and in repairing it. PNFS_OSD_ERR_EIO indicates the operation failed because the Object Storage Device experienced a failure trying to access the object. The most common source of these errors is media errors, but other internal errors might cause this. In this case, the metadata server should go examine the broken object more closely, hence it should be used as the default error code. PNFS_OSD_ERR_NOT_FOUND indicates the object ID specifies an object that does not exist on the Object Storage Device. PNFS_OSD_ERR_NO_SPACE indicates the operation failed because the Object Storage Device ran out of free capacity during the operation. PNFS_OSD_ERR_BAD_CRED indicates the security parameters are not valid. The primary cause of this is that the capability has expired, or the access policy tag (a.k.a, capability version number) has been changed to revoke capabilities. The client will need to return the layout and get a new one with fresh capabilities. PNFS_OSD_ERR_NO_ACCESS indicates the capability does not allow the requested operation. This should not occur in normal operation because the metadata server should give out correct capabilities, or none at all. PNFS_OSD_ERR_UNREACHABLE indicates the client did not complete the I/O operation at the Object Storage Device due to a communication failure. Whether the I/O operation was executed by the OSD or not is undetermined. PNFS_OSD_ERR_RESOURCE indicates the client did not issue the I/O operation due to a local problem on the initiator (i.e. client) side, e.g., when running out of memory. The client MUST guarantee that the OSD command was never dispatched to the OSD.

struct pnfs_osd_ioerr4 { pnfs_osd_objid4 component; length4 comp_offset; length4 comp_length; bool iswrite; pnfs_osd_errno4 errno; }; The pnfs_osd_ioerr4 structure is used to return error indications for objects that generated errors during data transfers. These are hints to the metadata server that there are problems with that object. For each error, "component", "comp_offset", and "comp_length" represent the object and byte range within the component object in which the error occurred. "iswrite" is set to "true" if the failed OSD operation was data modifying, and "errno" represents the type of error.

The layouthint4 type is defined in the NFSv4.1 draft as follows:

struct layouthint4 { layouttype4 loh_type; opaque loh_body<>; }; The layouthint4 structure is used by the client to pass in a hint about the type of layout it would like created for a particular file. If the loh_type layout type is LAYOUT4_OSD2_OBJECTS, then the loh_body opaque value is defined by the pnfs_osd_layouthint4 type.

union num_comps_hint4 switch (bool valid) { case TRUE: uint32_t num_comps; case FALSE: void; }; union stripe_unit_hint4 switch (bool valid) { case TRUE: length4 stripe_unit; case FALSE: void; }; union group_width_hint4 switch (bool valid) { case TRUE: uint32_t group_width; case FALSE: void; }; union group_depth_hint4 switch (bool valid) { case TRUE: uint32_t group_depth; case FALSE: void; }; union mirror_cnt_hint4 switch (bool valid) { case TRUE: uint32_t mirror_cnt; case FALSE: void; }; union raid_algorithm_hint4 switch (bool valid) { case TRUE: pnfs_osd_raid_algorithm4 raid_algorithm; case FALSE: void; }; struct pnfs_osd_layouthint4 { num_comps_hint4 num_comps_hint; stripe_unit_hint4 stripe_unit_hint; group_width_hint4 group_width_hint; group_depth_hint4 group_depth_hint; mirror_cnt_hint4 mirror_cnt_hint; raid_algorithm_hint4 raid_algorithm_hint; }; This type conveys hints for the desired data map. All parameters are optional so the client can give values for only the parameters it cares about, e.g. it can provide a hint for the desired number of mirrored components, regardless of the the raid algorithm selected for the file. The server should make an attempt to honor the hints but it can ignore any or all of them at its own discretion and without failing the respective create operation. The num_comps hint can be used to limit the total number of component objects comprising the file. All other hints correspond directly to the different fields of pnfs_osd_data_map4.

The pnfs layout operations operate on logical byte ranges. There is no requirement in the protocol for any relationship between byte ranges used in LAYOUTGET to acquire layouts and byte ranges used in CB_LAYOUTRECALL, LAYOUTCOMMIT, or LAYOUTRETURN. However, using OSD capabilities poses limitations on these operations since the capabilities associated with layout segments cannot be merged or split. The following guidelines should be followed for proper operation of object-based layouts.

In general, the object-based layout driver should keep track of each layout segment it got, keeping record of the segment's iomode, offset, and length. The server should allow the client to get multiple overlapping layout segments but is free to recall the layout to prevent overlap. In response to CB_LAYOUTRECALL, the client should return all layout segments matching the given iomode and overlapping with the recalled range. When returning the layouts for this byte range with LAYOUTRETURN the client MUST NOT return a sub-range of a layout segment it has; each LAYOUTRETURN sent MUST completely cover at least one outstanding layout segment. The server, in turn, should release any segment that exactly matches the clientid, iomode, and byte range given in LAYOUTRETURN. If no exact match is found then the server should release all layout segments matching the clientid and iomode and that are fully contained in the returned byte range. If none are found and the byte range is a subset of an outstanding layout segment with for the same clientid and iomode, then the client can be considered malfunctioning and the server SHOULD recall all layouts from this client to reset its state. If this behavior repeats the server SHOULD deny all LAYOUTGETs from this client.

LAYOUTCOMMIT is only used by object-based pNFS to convey modified attributes hints and/or to report I/O errors to the MDS. Therefore, the offset and length in LAYOUTCOMMIT4args are reserved for future use and should be set to 0. However, component byte ranges in the optional pnfs_osd_ioerr4 structure are used for recovering the object and MUST be set by the client to cover all failed I/O operations to the component.

The object-based metadata server should recall outstanding layouts in the following cases: When the file's security policy changes, i.e. ACLs or permission mode bits are set. When the file's aggregation map changes, rendering outstanding layouts invalid. When there are sharing conflicts. For example, the server will issue stripe aligned layout segments for RAID-5 objects. To prevent corruption of the file's parity, Multiple clients must not hold valid write layouts for the same stripes. An outstanding RW layout should be recalled when a conflicting LAYOUTGET is received from a different client for LAYOUTIOMODE4_RW and for a byte-range overlapping with the outstanding layout segment.

The metadata server can use the CB_RECALL_ANY callback operation to notify the client to return some or all of its layouts. The NFSv4.1 draft defines the following types:

const RCA4_TYPE_MASK_OBJ_LAYOUT_MIN = 8; const RCA4_TYPE_MASK_OBJ_LAYOUT_MAX = 11; struct CB_RECALL_ANY4args { uint32_t craa_objects_to_keep; bitmap4 craa_type_mask; }; Typically, CB_RECALL_ANY will be used to recall client state when the server needs to reclaim resources. The craa_type_mask bitmap specifies the type of resources that are recalled and the craa_objects_to_keep value specifies how many of the recalled objects the client is allowed to keep. The object-based layout type mask flags are defined as follows. They represent the iomode of the recalled layouts. In response, the client SHOULD return layouts of the recalled iomode that it needs the least, keeping at most craa_objects_to_keep object-based layouts.

const PNFS_OSD_RCA4_TYPE_MASK_READ = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN; const PNFS_OSD_RCA4_TYPE_MASK_RW = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN+1; const PNFS_OSD_RCA4_TYPE_MASK_ANY = RCA4_TYPE_MASK_OBJ_LAYOUT_MIN+2; The PNFS_OSD_RCA4_TYPE_MASK_READ flag notifies the client to return layouts of iomode LAYOUTIOMODE4_READ. Similarly, the PNFS_OSD_RCA4_TYPE_MASK_RW flag notifies the client to return layouts of iomode LAYOUTIOMODE4_RW. The PNFS_OSD_RCA4_TYPE_MASK_ANY flag notifies the client to return layouts of either iomode.

In cases where clients are uncommunicative and their lease has expired or when clients fail to return recalled layouts in a timely manner the server MAY revoke client layouts and/or device address mappings and reassign these resources to other clients. To avoid data corruption, the metadata server MUST fence off the revoked clients from the respective objects as described in .

The pNFS extension partitions the NFSv4 file system protocol into two parts, the control path and the data path (storage protocol). The control path contains all the new operations described by this extension; all existing NFSv4 security mechanisms and features apply to the control path. The combination of components in a pNFS system is required to preserve the security properties of NFSv4 with respect to an entity accessing data via a client, including security countermeasures to defend against threats that NFSv4 provides defenses for in environments where these threats are considered significant. The metadata server enforces the file access-control policy at LAYOUTGET time. The client should use suitable authorization credentials for getting the layout for the requested iomode (READ or RW) and the server verifies the permissions and ACL for these credentials, possibly returning NFS4ERR_ACCESS if the client is not allowed the requested iomode. If the LAYOUTGET operation succeeds the client receives, as part of the layout, a set of object capabilities allowing it I/O access to the specified objects corresponding to the requested iomode. When the client acts on I/O operations on behalf of its local users it MUST authenticate and authorize the user by issuing respective OPEN and ACCESS calls to the metadata server, similarly to having NFSv4 data delegations. If access is allowed the client uses the corresponding (READ or RW) capabilities to perform the I/O operations at the object-storage devices. When the metadata server receives a request to change file's permissions or ACL it SHOULD recall all layouts for that file and it MUST change the capability version attribute on all objects comprising the file to implicitly invalidate any outstanding capabilities before committing to the new permissions and ACL. Doing this will ensure that clients re-authorize their layouts according to the modified permissions and ACL by requesting new layouts. Recalling the layouts in this case is courtesy of the server intended to prevent clients from getting an error on I/Os done after the capability version changed. The object storage protocol MUST implement the security aspects described in version 1 of the T10 OSD protocol definition. The standard defines four security methods: NOSEC, CAPKEY, CMDRSP, and ALLDATA. To provide minimum level of security allowing verification and enforcement of the server access control policy using the layout security credentials, the NOSEC security method MUST NOT be used for I/O operation. It MAY only be used to get the System ID attribute when the metadata server provided only the OSD name with the device address. The remainder of this section gives an overview of the security mechanism described in that standard. The goal is to give the reader a basic understanding of the object security model. Any discrepancies between this text and the actual standard are obviously to be resolved in favor of the OSD standard.

There are three main data types associated with object security: a capability, a credential, and security parameters. The capability is a set of fields that specifies an object and what operations can be performed on it. A credential is a signed capability. Only a security manager that knows the secret device keys can correctly sign a capability to form a valid credential. In pNFS, the file server acts as the security manager and returns signed capabilities (i.e., credentials) to the pNFS client. The security parameters are values computed by the issuer of OSD commands (i.e., the client) that prove they hold valid credentials. The client uses the credential as a signing key to sign the requests it makes to OSD, and puts the resulting signatures into the security_parameters field of the OSD command. The object storage device uses the secret keys it shares with the security manager to validate the signature values in the security parameters. The security types are opaque to the generic layers of the pNFS client. The credential contents are defined as opaque within the pnfs_osd_object_cred4 type. Instead of repeating the definitions here, the reader is referred to section 4.9.2.2 of the OSD standard.

The object storage protocol relies on a cryptographically secure capability to control accesses at the object storage devices. Capabilities are generated by the metadata server, returned to the client, and used by the client as described below to authenticate their requests to the Object Storage Device (OSD). Capabilities therefore achieve the required access and open mode checking. They allow the file server to define and check a policy (e.g., open mode) and the OSD to enforce that policy without knowing the details (e.g., user IDs and ACLs). Since capabilities are tied to layouts, and since they are used to enforce access control, when the file ACL or mode changes the outstanding capabilities MUST be revoked to enforce the new access permissions. The server SHOULD recall layouts to allow clients to gracefully return their capabilities before the access permissions change. Each capability is specific to a particular object, an operation on that object, a byte range w/in the object (in OSDv2), and has an explicit expiration time. The capabilities are signed with a secret key that is shared by the object storage devices (OSD) and the metadata managers. Clients do not have device keys so they are unable to forge the signatures in the security parameters. The combination of a capability, the OSD system id, and a signature is called a "credential" in the OSD specification. The details of the security and privacy model for Object Storage are defined in the T10 OSD standard. The following sketch of the algorithm should help the reader understand the basic model. LAYOUTGET returns a CapKey and a Cap which, together with the OSD SystemID, are also called a credential. It is a capability and a signature over that capability and the SystemID. The OSD Standard refers to the CapKey as the "Credential integrity check value" and to the ReqMAC as the "Request integrity check value".

(Cap, SystemID) Credential = {Cap, SystemID, CapKey} ]]> The client uses CapKey to sign all the requests it issues for that object using the respective Cap. In other words, the Cap appears in the request to the storage device, and that request is signed with the CapKey as follows:

(Req, ReqNonce) Request = {Cap, Req, ReqNonce, ReqMAC} ]]> The following is sent to the OSD: {Cap, Req, ReqNonce, ReqMAC}. The OSD uses the SecretKey it shares with the metadata server to compare the ReqMAC the client sent with a locally computed value:

(Cap, SystemID) LocalReqMAC = MAC(Req, ReqNonce) ]]> and if they match the OSD assumes that the capabilities came from an authentic metadata server and allows access to the object, as allowed by the Cap.

Note that if the server LAYOUTGET reply, holding CapKey and Cap, is snooped by another client, it can be used to generate valid OSD requests (within the Cap access restrictions). To provide the required privacy requirements for the capability key returned by LAYOUTGET, the GSS-API can be used, e.g. by using the RPCSEC_GSS privacy method to send the LAYOUTGET operation or by using the SSV key to encrypt the capability_key using the GSS_Wrap() function. Two general ways to provide privacy in the absence of GSS-API that are independent of NFSv4 are either an isolated network such as a VLAN or a secure channel provided by IPsec.

At any time, the metadata server may invalidate all outstanding capabilities on an object by changing its POLICY ACCESS TAG attribute. The value of the POLICY ACCESS TAG is part of a capability, and it must match the state of the object attribute. If they do not match, the OSD rejects accesses to the object with the sense key set to ILLEGAL REQUEST and an additional sense code set to INVALID FIELD IN CDB. When a client attempts to use a capability and is rejected this way, it should issue a LAYOUTCOMMIT for the object and specify PNFS_OSD_BAD_CRED in the ioerr parameter. The client may elect to issue a compound LAYOUTRETURN/LAYOUTGET (or LAYOUTCOMMIT/LAYOUTRETURN/LAYOUTGET) to attempt to fetch a refreshed set of capabilities. The metadata server may elect to change the access policy tag on an object at any time, for any reason (with the understanding that there is likely an associated performance penalty, especially if there are outstanding layouts for this object). The metadata server MUST revoke outstanding capabilities when any one of the following occurs: the permissions on the object change, a conflicting mandatory byte-range lock is granted, or a layout is revoked and reassigned to another client A pNFS client will typically hold one layout for each byte range for either READ or READ/WRITE. The client's credentials are checked by the metadata server at LAYOUTGET time and it is the client's responsibility to enforce access control among multiple users accessing the same file. It is neither required nor expected that the pNFS client will obtain a separate layout for each user accessing a shared object. The client SHOULD use OPEN and ACCESS calls to check user permissions when performing I/O so that the server's access control policies are correctly enforced. The result of the ACCESS operation may be cached while the client holds a valid layout as the server is expected to recall layouts when the file's access permissions or ACL change.

As described in the NFSv4.1 draft, new layout type numbers will be requested from IANA. This document defines the protocol associated with the existing layout type number, LAYOUT4_OSD2_OBJECTS, and it requires no further actions for IANA.

SCSI Object-Based Storage Device Commands SNIA T10/1355-D Key words for use in RFCs to Indicate Requirement Levels Harvard University

1350 Mass. Ave. Cambridge MA 02138 - +1 617 495 3864 sob@harvard.edu

XDR: External Data Representation Standard Network Appliance, Inc. Network File System (NFS) version 4 Protocol Sun Microsystems, Inc. Sun Microsystems, Inc. Sun Microsystems, Inc. Sun Microsystems, Inc. Hummingbird, Ltd. Network Appliance, Inc. Network Appliance, Inc. Generic Security Service Application Program Interface Version 2, Update 1 RSA Laboratories

20 Crosby Drive Bedford MA 01730 US +1 781 687 7817 jlinn@rsasecurity.com

The Generic Security Service Application Program Interface (GSS-API), Version 2, as defined in [RFC-2078], provides security services to callers in a generic fashion, supportable with a range of underlying mechanisms and technologies and hence allowing source-level portability of applications to different environments. This specification defines GSS-API services and primitives at a level independent of underlying mechanism and programming language environment, and is to be complemented by other, related specifications: documents defining specific parameter bindings for particular language environments documents defining token formats, protocols, and procedures to be implemented in order to realize GSS-API services atop particular security mechanisms This memo obsoletes [RFC-2078], making specific, incremental changes in response to implementation experience and liaison requests. It is intended, therefore, that this memo or a successor version thereto will become the basis for subsequent progression of the GSS-API specification on the standards track. NFSv4 Minor Version 1 Sun Microsystems, Inc. Network Appliance, Inc. Network Appliance, Inc. SCSI Object-Based Storage Device Commands -2 (OSD-2) SNIA T10/1729-D Security Architecture for the Internet Protocol BBN Technologies BBN Technologies

Todd Pisek was a co-editor of the initial drafts for this document.

--------------010303050605070809060207 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 --------------010303050605070809060207-- From MereditheaseCastle@achievecareer.com Thu Sep 06 16:09:59 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITNfr-00086C-EM for nfsv4-archive@lists.ietf.org; Thu, 06 Sep 2007 16:09:59 -0400 Received: from c226-156.i03-9.onvol.net ([213.217.226.156] helo=user4a7754114f) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1ITNfp-00058N-CS for nfsv4-archive@lists.ietf.org; Thu, 06 Sep 2007 16:09:57 -0400 Message-ID: <12b7001c7f0c1$dee19ab0$6502a8c0@user4a7754114f> From: "Freda Castle" To: Subject: Fwd: Thanks, we are accepting your company loan request Date: Thu, 6 Sep 2007 22:07:57 -0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_12B6C_01C7F0C1.DEE19AB0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962 X-Spam-Score: 4.2 (++++) X-Scan-Signature: cab78e1e39c4b328567edb48482b6a69 ------=_NextPart_000_12B6C_01C7F0C1.DEE19AB0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable If you have your own business and need IMMEDIATE money to spend ANY way = you like or wish Extra money to give your business a boost or want A low = interest loan - NO STRINGS ATTACHED, here is our deal we can offer you = TONIGHT (hurry, this tender will expire THIS NIGHT): $65,000+ loan Hurry, when our deal is gone, it is gone. Simply Call Us Free on = 877-482-4954 ------=_NextPart_000_12B6C_01C7F0C1.DEE19AB0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

If you have your own business and need = IMMEDIATE=20 money to spend ANY way you like or wish Extra money to give your = business a=20 boost or want A low interest loan - NO STRINGS ATTACHED, here is our = deal we=20 can offer you TONIGHT (hurry, this tender will expire THIS = NIGHT):

=20 =20

$65,000+ loan

Hurry, when our deal is gone, it is = gone. Simply=20 Call Us Free on 877-482-4954

------=_NextPart_000_12B6C_01C7F0C1.DEE19AB0-- From Nike.Tehansky@305design.com Thu Sep 06 17:15:48 2007 Return-path: Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITOhY-0005DG-4q for nfsv4-archive@lists.ietf.org; Thu, 06 Sep 2007 17:15:48 -0400 Received: from fis50.internetdsl.tpnet.pl ([83.13.226.50]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ITOhW-0003oR-Dt for nfsv4-archive@lists.ietf.org; Thu, 06 Sep 2007 17:15:48 -0400 Received: from damcio by 305design.com with ASMTP id 400D7536 for ; Thu, 6 Sep 2007 23:16:09 +0200 Received: from damcio ([185.149.147.5]) by 305design.com with ESMTP id 4A524EA54407 for ; Thu, 6 Sep 2007 23:16:09 +0200 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 6 Sep 2007 23:15:46 +0200 To: nfsv4-archive@lists.ietf.org From: "Nike Tehansky" Subject: ankauppa Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.4 (+) X-Scan-Signature: 8ac499381112328dd60aea5b1ff596ea Wazzup Tehansky Get More Powerful erections - Develop 'rock hard' erections, each and every time no matter your age! pipo Sider http://seslcorp.com/ From nfsv4-bounces@ietf.org Thu Sep 06 22:47:08 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITTqQ-0006dP-O5; Thu, 06 Sep 2007 22:45:18 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITTqP-0006IX-2K for nfsv4@ietf.org; Thu, 06 Sep 2007 22:45:17 -0400 Received: from brmea-mail-3.sun.com ([192.18.98.34]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ITTqO-00085u-96 for nfsv4@ietf.org; Thu, 06 Sep 2007 22:45:16 -0400 Received: from fe-amer-05.sun.com ([192.18.108.179]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l872jFvi025959 for ; Fri, 7 Sep 2007 02:45:15 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNZ007018GRWN00@mail-amer.sun.com> (original mail from Spencer.Shepler@Sun.COM) for nfsv4@ietf.org; Thu, 06 Sep 2007 20:45:15 -0600 (MDT) Received: from [10.1.194.251] (sca-ea-proxy-2.Sun.COM [192.18.43.27]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNZ00H9C8ZEGL40@mail-amer.sun.com> for nfsv4@ietf.org; Thu, 06 Sep 2007 20:45:15 -0600 (MDT) Date: Thu, 06 Sep 2007 21:45:10 -0500 From: Spencer Shepler To: nfsv4@ietf.org Message-id: <5787AFE2-4C37-4CAD-9E4A-35E54FC440EA@Sun.COM> MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; format=flowed; charset=US-ASCII Content-transfer-encoding: 7BIT X-Spam-Score: 0.0 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Subject: [nfsv4] Timeline for NFSv4.1 I-D updates and last call preparation X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org To set some expectations for updates to the NFSv4.1 internet draft and review cycle, I am passing along the rough timeline the editors and working group co-chairs are working towards. The basis of this timeline was discussed at the Chicago IETF meeting and recorded in the notes so I hope that it is not a complete surprise. Please review this in the mindset of doing a full document review in preparation for the working group last call starting with Draft 17. Spencer Sept 14th - Draft 14 Contains updates to namespace sections based on formal reviews, final pNFS issues, and sessions review (EXCHANGE_ID, etc.) Oct 5th - Draft 15 Contains updates from the locking, state, delegations reviews and any issues from the errors/ops review Oct 19th - Draft 16 Contains general issues and those items raised/discussed during the bakeathon. With Draft 16, we should be "ready" for working group last call. Although we will not start the official timer for last call, the intent is that the working group treat it as such given the size of the document. Nov 12 - Draft 17 Any issues found in review and last call version _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From Rehktman@amcitesting.com Fri Sep 07 07:09:45 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITbib-0002YI-Gm for nfsv4-archive@lists.ietf.org; Fri, 07 Sep 2007 07:09:45 -0400 Received: from host200-175-dynamic.16-87-r.retail.telecomitalia.it ([87.16.175.200] helo=host37-175-dynamic.6-87-r.retail.telecomitalia.it) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ITbia-0001Wg-TH for nfsv4-archive@lists.ietf.org; Fri, 07 Sep 2007 07:09:45 -0400 Received: by 10.38.134.140 with SMTP id wxioSAeBtGamD; Fri, 7 Sep 2007 13:09:54 +0200 (GMT) Received: by 192.168.200.127 with SMTP id DMqrabuDCtkrJS.3272592073380; Fri, 7 Sep 2007 13:09:52 +0200 (GMT) Message-ID: <000901c7f13f$9b918f30$25af0657@PC> From: "Art Rehktman" To: Subject: otiohnir Date: Fri, 7 Sep 2007 13:09:49 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0007_01C7F150.5F1A5F30" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 4.2 (++++) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 ------=_NextPart_000_0007_01C7F150.5F1A5F30 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.hammned.com/ How it is going Rehktman Which are the Best Penis Enlargement pills.... These ones are.. Arick Matza ------=_NextPart_000_0007_01C7F150.5F1A5F30 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.hammned.com/

How it is going Rehktman

Which are the Best Penis Enlargement pills.... = These=20 ones are..

Arick Matza

------=_NextPart_000_0007_01C7F150.5F1A5F30-- From leeroy756@ikefast.com Fri Sep 07 08:47:13 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITdEv-0003xb-Sw for nfsv4-archive@lists.ietf.org; Fri, 07 Sep 2007 08:47:13 -0400 Received: from host55-205-static.63-82-b.business.telecomitalia.it ([82.63.205.55]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ITdEu-00049j-V3 for nfsv4-archive@lists.ietf.org; Fri, 07 Sep 2007 08:47:13 -0400 Received: from agenti ([198.185.59.160] helo=agenti) by host55-205-static.63-82-b.business.telecomitalia.it ( sendmail 8.13.3/8.13.1) with esmtpa id 1fjicv-000VLJ-tL for nfsv4-archive@lists.ietf.org; Fri, 7 Sep 2007 14:49:08 +0200 Message-ID: <000a01c7f14d$63d66800$37cd3f52@agenti> From: "leeroy brister" To: Subject: 4472603 Date: Fri, 7 Sep 2007 14:48:29 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0003_01C7F15E.275F3800" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 2.0 (++) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 ------=_NextPart_000_0003_01C7F15E.275F3800 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://havisue.com/ Yo brister Ladies can sence a confident man, and they like it. Ofoffon Rahimi ------=_NextPart_000_0003_01C7F15E.275F3800 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://havisue.com/

Yo brister

Ladies can sence a confident man, and they = like it.

Ofoffon Rahimi

------=_NextPart_000_0003_01C7F15E.275F3800-- From Elbornejps@trendydog.net Sat Sep 08 03:07:32 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITuPk-0000Xn-Mw for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 03:07:32 -0400 Received: from cnv252.neoplus.adsl.tpnet.pl ([83.31.175.252]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ITuPk-00063z-2P for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 03:07:32 -0400 Received: from marcinsklep ([181.160.86.5] helo=marcinsklep) by cnv252.neoplus.adsl.tpnet.pl ( sendmail 8.13.3/8.13.1) with esmtpa id 1NXrQq-000YPU-Qf for nfsv4-archive@lists.ietf.org; Sat, 8 Sep 2007 09:07:51 +0200 Message-ID: <000201c7f1e6$f17de400$fcaf1f53@marcinsklep> From: "Jeighms Elborne" To: Subject: talit{rt Date: Sat, 8 Sep 2007 09:07:39 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C7F1F7.B506B400" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 3.4 (+++) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 ------=_NextPart_000_0005_01C7F1F7.B506B400 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable http://www.heabank.com/ Yo nfsv4-archive Did you ever ask yourself is my penis big enough???? Judythe Brauch ------=_NextPart_000_0005_01C7F1F7.B506B400 Content-Type: text/html; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable

http://www.heabank.com/

Yo nfsv4-archive

Did you ever ask yourself is my penis big = enough????

Judythe Brauch

------=_NextPart_000_0005_01C7F1F7.B506B400-- From Dallas_Omark@lakemichigansamplerguild.org Sat Sep 08 08:57:02 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ITzry-0005jh-IB for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 08:57:02 -0400 Received: from brc217.neoplus.adsl.tpnet.pl ([83.29.96.217]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1ITzrx-00071C-MJ for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 08:57:02 -0400 Received: from misio by lakemichigansamplerguild.org with ASMTP id B2AEBE7A for ; Sat, 8 Sep 2007 14:57:36 +0200 Received: from misio ([115.107.121.48]) by lakemichigansamplerguild.org with ESMTP id B8E254480E3E for ; Sat, 8 Sep 2007 14:57:36 +0200 Message-ID: <000c01c7f217$bd92c2b0$d9601d53@misio> From: "Dallas Omark" To: Subject: thginhtr Date: Sat, 8 Sep 2007 14:56:58 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0006_01C7F228.811B92B0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 3.8 (+++) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 ------=_NextPart_000_0006_01C7F228.811B92B0 Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: quoted-printable http://www.hresal.com/ Wassup nfsv4-archive Say hello to my little friend, well big friend now du jobs ------=_NextPart_000_0006_01C7F228.811B92B0 Content-Type: text/html; charset="windows-1250" Content-Transfer-Encoding: quoted-printable

http://www.hresal.com/

Wassup nfsv4-archive

Say hello to my little friend, well big friend = now

du jobs

------=_NextPart_000_0006_01C7F228.811B92B0-- From Jaye.lavalley@customawardsandtrophies.com Sat Sep 08 09:29:35 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IU0NT-0003F3-DI for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 09:29:35 -0400 Received: from chello062179020063.chello.pl ([62.179.20.63]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IU0NQ-0008O4-SR for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 09:29:33 -0400 Received: from computex-h7qx22 ([164.198.26.86] helo=computex-h7qx22) by chello062179020063.chello.pl ( sendmail 8.13.3/8.13.1) with esmtpa id 1cwNAF-000JBM-Jl for nfsv4-archive@lists.ietf.org; Sat, 8 Sep 2007 15:31:01 +0200 Message-ID: <000501c7f21c$6ace9cc0$3f14b33e@computexh7qx22> From: "Jaye lavalley" To: Subject: gnikahcs Date: Sat, 8 Sep 2007 15:30:26 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0008_01C7F22D.2E576CC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 3.4 (+++) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 ------=_NextPart_000_0008_01C7F22D.2E576CC0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable http://hresal.com/ Wazzup nfsv4-archive oh yes, intercourse is alot better now im BIGGER in size Marjo guy ------=_NextPart_000_0008_01C7F22D.2E576CC0 Content-Type: text/html; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable

http://hresal.com/

Wazzup nfsv4-archive

oh yes, intercourse is alot better now im = BIGGER in size

Marjo guy

------=_NextPart_000_0008_01C7F22D.2E576CC0-- From Kiarash.Nader@easternshorelawncare.com Sat Sep 08 19:44:16 2007 Return-path: Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IU9yK-0001F9-CQ for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 19:44:16 -0400 Received: from ti300710a340-1207.bb.online.no ([88.88.92.183]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IU9yI-0006WQ-Ms for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 19:44:16 -0400 Received: from oso by easternshorelawncare.com with ASMTP id 8533F14C for ; Sun, 9 Sep 2007 01:44:41 +0200 Received: from oso ([115.198.163.6]) by easternshorelawncare.com with ESMTP id 2FCB373EF3A7 for ; Sun, 9 Sep 2007 01:44:41 +0200 Message-ID: <000301c7f272$28f18960$b75c5858@oso> From: "Kiarash Nader" To: Subject: folium Date: Sun, 9 Sep 2007 01:44:13 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0006_01C7F282.EC7A5960" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 1.6 (+) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 ------=_NextPart_000_0006_01C7F282.EC7A5960 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.hollanyc.com/ Yo Nader check out this website, limited offers (MEN ONLY) Tricia decca ------=_NextPart_000_0006_01C7F282.EC7A5960 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.hollanyc.com/

Yo Nader

check out this website, limited offers (MEN = ONLY)

Tricia decca

------=_NextPart_000_0006_01C7F282.EC7A5960-- From Cavanaugh.Boueke@the0123.com Sat Sep 08 19:56:08 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUA9o-0005ez-Dh for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 19:56:08 -0400 Received: from [88.245.151.124] (helo=[88.245.151.124]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUA9n-0003ma-Iu for nfsv4-archive@lists.ietf.org; Sat, 08 Sep 2007 19:56:08 -0400 Received: from masa02 by the0123.com with ASMTP id 64A426CD for ; Sun, 9 Sep 2007 02:56:26 +0300 Received: from masa02 ([136.127.80.33]) by the0123.com with ESMTP id 971C83E53CD1 for ; Sun, 9 Sep 2007 02:56:26 +0300 Message-ID: <9CBE76D3.5E895B03@the0123.com> Date: Sun, 9 Sep 2007 02:56:14 +0300 From: "Cavanaugh Boueke" User-Agent: Thunderbird 1.5.0.10 (Windows/20070221) MIME-Version: 1.0 To: nfsv4-archive@lists.ietf.org Subject: aines Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 2.1 (++) X-Scan-Signature: 8ac499381112328dd60aea5b1ff596ea Hi bro Boueke winner of the 2007 herbal gold medal Len Khamsaysouk http://hrlyi.com/ From Ricard@WALKENGR.COM Sun Sep 09 09:30:43 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUMs7-0007Dr-Tn for nfsv4-archive@lists.ietf.org; Sun, 09 Sep 2007 09:30:43 -0400 Received: from [194.105.50.22] (helo=[194.105.50.22]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUMs6-0002J2-7v for nfsv4-archive@lists.ietf.org; Sun, 09 Sep 2007 09:30:43 -0400 Received: by 10.200.96.209 with SMTP id BCxUEpKLQRQaQ; Sun, 9 Sep 2007 15:36:41 +0200 (GMT) Received: by 192.168.199.49 with SMTP id XrCwwRpdoQTuTy.4694626694159; Sun, 9 Sep 2007 15:36:39 +0200 (GMT) Message-ID: <000601c7f2e6$71b86be0$163269c2@vigile> From: "Spencer Ricard" To: Subject: kacyou Date: Sun, 9 Sep 2007 15:36:36 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0008_01C7F2F7.35413BE0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 4.2 (++++) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 ------=_NextPart_000_0008_01C7F2F7.35413BE0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://www.sloebers.com/ Hi bro Spencer I started to notice that my penis had grown at least an inch Oren Romanosky ------=_NextPart_000_0008_01C7F2F7.35413BE0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://www.sloebers.com/

Hi bro Spencer

I started to notice that my penis had grown at = least an inch

Oren Romanosky

------=_NextPart_000_0008_01C7F2F7.35413BE0-- From nfsv4-bounces@ietf.org Sun Sep 09 22:59:00 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZSU-0004Hu-CY; Sun, 09 Sep 2007 22:57:06 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZQS-0001xd-G1 for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:00 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUZQR-0005st-FN for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:00 -0400 X-IronPort-AV: E=Sophos;i="4.20,228,1186383600"; d="scan'208";a="101983412" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 09 Sep 2007 19:54:58 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l8A2svn7020117 for ; Sun, 9 Sep 2007 19:54:58 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:54:57 -0700 Received: from exsvl03.hq.netapp.com ([10.56.8.64]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:54:57 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sun, 9 Sep 2007 19:54:16 -0700 Message-ID: <0B79955C903FA0438DDA4FFCC2BA521B02994949@exsvl03.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part I) Thread-Index: AcfeTOk1jUuB1FA+TA6l9LGNVoZQjgAKSGjQAE6QHJAEv4ByUA== From: "Erasani, Pranoop" To: X-OriginalArrivalTime: 10 Sep 2007 02:54:57.0515 (UTC) FILETIME=[F8CAABB0:01C7F355] X-Spam-Score: 0.0 (/) X-Scan-Signature: 850245b51c39701e2700a112f3032caa X-Mailman-Approved-At: Sun, 09 Sep 2007 22:57:05 -0400 Subject: [nfsv4] NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part I) X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org Please find below the review comments captured from the first review session scheduled for inspection of "Exchange ID and Destroy Client ID" related chapters. #Date# August 21, 2007, 12 - 2 PM PST #Participants# Mike Eisler - Network Appliance (Moderator/Author) Spencer Shepler - Sun Microsystems (Author) Dave Noveck - Sun Microsystems (Inspector) Ricardo Labiaga - Network Appliance (Reader) Saadia Khan - Network Appliance (Inspector) Pranoop Erasani - Network Appliance (Scribe/Inspector) Robert gordon - Sun Microsystems (Inspector) Brent Welsch - Panasas (Inspector) Benny Halevy - Panasas (Inspector) #Review Comments# Line No. Raised By Comment -------- --------- ------- 868-873 Labiaga/Erasani Needs better wording. The two-tiered approach adds to more confusion. General Gordon An application level program can be a client too. Using "reboot" does not make sense in that context. s/reboot/re-initialization. 1196 Labiaga Does NFSv4 represent NFSv4 and NFSv4.1. Suggestion: s/NFSv4/NFSv4.1 1331-1335 Noveck,Khan, This exists based on the assumption that NFSv4.0 and NFSv4.1 can talk to each other. Erasani Suggestion: Remove these and make it explicit that both NFSv4.0 and NFSv4.1 are separate. 1386 Khan Mention the appropriate flag here (sr_status_flags). 1408 Khan Should we mention if state cleanup is Ok? 1408 Noveck Can we desFrom nfsv4-bounces@ietf.org Sun Sep 09 22:59:00 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZSU-0004Hu-CY; Sun, 09 Sep 2007 22:57:06 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZQS-0001xd-G1 for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:00 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUZQR-0005st-FN for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:00 -0400 X-IronPort-AV: E=Sophos;i="4.20,228,1186383600"; d="scan'208";a="101983412" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 09 Sep 2007 19:54:58 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l8A2svn7020117 for ; Sun, 9 Sep 2007 19:54:58 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:54:57 -0700 Received: from exsvl03.hq.netapp.com ([10.56.8.64]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:54:57 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sun, 9 Sep 2007 19:54:16 -0700 Message-ID: <0B79955C903FA0438DDA4FFCC2BA521B02994949@exsvl03.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part I) Thread-Index: AcfeTOk1jUuB1FA+TA6l9LGNVoZQjgAKSGjQAE6QHJAEv4ByUA== From: "Erasani, Pranoop" To: X-OriginalArrivalTime: 10 Sep 2007 02:54:57.0515 (UTC) FILETIME=[F8CAABB0:01C7F355] X-Spam-Score: 0.0 (/) X-Scan-Signature: 850245b51c39701e2700a112f3032caa X-Mailman-Approved-At: Sun, 09 Sep 2007 22:57:05 -0400 Subject: [nfsv4] NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part I) X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org Please find below the review comments captured from the first review session scheduled for inspection of "Exchange ID and Destroy Client ID" related chapters. #Date# August 21, 2007, 12 - 2 PM PST #Participants# Mike Eisler - Network Appliance (Moderator/Author) Spencer Shepler - Sun Microsystems (Author) Dave Noveck - Sun Microsystems (Inspector) Ricardo Labiaga - Network Appliance (Reader) Saadia Khan - Network Appliance (Inspector) Pranoop Erasani - Network Appliance (Scribe/Inspector) Robert gordon - Sun Microsystems (Inspector) Brent Welsch - Panasas (Inspector) Benny Halevy - Panasas (Inspector) #Review Comments# Line No. Raised By Comment -------- --------- ------- 868-873 Labiaga/Erasani Needs better wording. The two-tiered approach adds to more confusion. General Gordon An application level program can be a client too. Using "reboot" does not make sense in that context. s/reboot/re-initialization. 1196 Labiaga Does NFSv4 represent NFSv4 and NFSv4.1. Suggestion: s/NFSv4/NFSv4.1 1331-1335 Noveck,Khan, This exists based on the assumption that NFSv4.0 and NFSv4.1 can talk to each other. Erasani Suggestion: Remove these and make it explicit that both NFSv4.0 and NFSv4.1 are separate. 1386 Khan Mention the appropriate flag here (sr_status_flags). 1408 Khan Should we mention if state cleanup is Ok? 1408 Noveck Can we destroy clientid when associated sessions are still alive? 1408 Noveck In general, we need to differentiate between lock and stateid. (Dave: Lock can't be cleaned up, but stateid can be cleaned) 1433 Welsch During EXCHANGEID operation itself we can resolve the client owner conflicts in terms of State and unexpired lease. 1438 Khan MUST NOT -> Need to prune down the negatives. 1432 Noveck,Eisler, Referred to having state. But, what does it mean? Clientid? Session? Lock? Stateid? Suggestion: clientid is state. Gordon 1466 Labiaga Lacks in-depth overview and jumps into conclusions on SSV. 1479 Labiaga Remove double negatives 1483 Erasani Elaborate update properties -> too many to list here? 22744 Halevy Sesion stateid is missing 22815 Halevy/Labiaga Hash function is not *NULL* 22895 Noveck Should we put timeout on the expiry of client owner ID's? 23063 Khan "or" is missing. (Talk to saadia) 23106 Noveck BIND_PRINV_STATEID-> Changing the bit means what? Eisler: Furute ones only. 23106 Gordon Not clear whethere this bit is set on both MDS and DS? What are the options for the server?=09 23135 Noveck, Eisler Refer "pNFS layout" from here. 23224 Eisler It's not clear how the bit map is mapped to operation values. 23162 Gordon Is "pNFS use profile" a drag? 23162 Welsch Can a client have different machine credentials tied to different sessions for the same clientid? 23246/23259 Labiaga If empty array, what's the appropriate error? (Eisler: INVAL) 23272 Eisler If ssp_window is 0 - invalid, 1 - one version of SSV support at any one time. 23280 Labiaga Is "0" valid? 23276 Eisler s/are using SSVs/are using the SSV OR s/are using SSVs/are using the version of the SSV. 23278 Noveck Why use slotid's? Meeting ended with review upto Line No. 23327. Apologies in advance, if I have associated a wrong name with the comment. Thanks Pranoop -----Original Message----- From: Eisler, Michael Sent: Tuesday, August 14, 2007 6:25 AM To: 'Spencer Shepler'; Van Belleghem,Audrey; bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey, Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com; andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM; trond.myklebust@fys.uio.no; Noveck, Dave; Khan, Saadia; Erasani, Pranoop Subject: RE: Kickoff: Exchange ID and destroy Client ID Here are the lines of the spec, http://www.nfsv4-editor.org/draft-13/draft-ietf-nfsv4-minorversion1-13-l n.txt that should be inspected (or simply read as noted below). Definitions 815-849 866-844 Client and Server Owners 1193-1521 Trunking 2263-2424 - Previously inspected. Will not be re-inspected but useful for background material. State Protection Model 3315-3722 - Previously inspected. Will not be re-inspected but useful for background material. EXCHANGE_ID 22726-23661 SET_SSV 25681-25786 DESTROY_CLIENTIID 25995-26034=20 > -----Original Message----- > From: Spencer Shepler [mailto:Spencer.Shepler@Sun.COM] > Sent: Tuesday, August 14, 2007 1:25 AM > To: Van Belleghem,Audrey > Cc: bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey,=20 > Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com;=20 > andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM;=20 > trond.myklebust@fys.uio.no; Noveck, Dave; Eisler, Michael > Subject: Re: Kickoff: Exchange ID and destroy Client ID >=20 >=20 > On Jul 30, 2007, at 4:28 PM, Van Belleghem,Audrey wrote: >=20 > > Hi, > > > > Thanks to those of you who have already participated in > portions of > > the NFSv4.1 specification review. For those that don't know me, I'm > > a program manager working for Brian Pawlowski in the > CTO Office > > of Network Appliance. Brian is co-chair of the IETF's NFSv4 working > > group which has submitted an Internet draft titled "NFSv4 Minor=20 > > Version 1". > > > > The next chapters we'll be reviewing are Exchange ID and Destroy=20 > > Client ID. We'll do those together. > > > > The process will work as follows: > > > > 7/30/07- Call for reviewers (please forward this messagetroy clientid when associated sessions are still alive? 1408 Noveck In general, we need to differentiate between lock and stateid. (Dave: Lock can't be cleaned up, but stateid can be cleaned) 1433 Welsch During EXCHANGEID operation itself we can resolve the client owner conflicts in terms of State and unexpired lease. 1438 Khan MUST NOT -> Need to prune down the negatives. 1432 Noveck,Eisler, Referred to having state. But, what does it mean? Clientid? Session? Lock? Stateid? Suggestion: clientid is state. Gordon 1466 Labiaga Lacks in-depth overview and jumps into conclusions on SSV. 1479 Labiaga Remove double negatives 1483 Erasani Elaborate update properties -> too many to list here? 22744 Halevy Sesion stateid is missing 22815 Halevy/Labiaga Hash function is not *NULL* 22895 Noveck Should we put timeout on the expiry of client owner ID's? 23063 Khan "or" is missing. (Talk to saadia) 23106 Noveck BIND_PRINV_STATEID-> Changing the bit means what? Eisler: Furute ones only. 23106 Gordon Not clear whethere this bit is set on both MDS and DS? What are the options for the server?=09 23135 Noveck, Eisler Refer "pNFS layout" from here. 23224 Eisler It's not clear how the bit map is mapped to operation values. 23162 Gordon Is "pNFS use profile" a drag? 23162 Welsch Can a client have different machine credentials tied to different sessions for the same clientid? 23246/23259 Labiaga If empty array, what's the appropriate error? (Eisler: INVAL) 23272 Eisler If ssp_window is 0 - invalid, 1 - one version of SSV support at any one time. 23280 Labiaga Is "0" valid? 23276 Eisler s/are using SSVs/are using the SSV OR s/are using SSVs/are using the version of the SSV. 23278 Noveck Why use slotid's? Meeting ended with review upto Line No. 23327. Apologies in advance, if I have associated a wrong name with the comment. Thanks Pranoop -----Original Message----- From: Eisler, Michael Sent: Tuesday, August 14, 2007 6:25 AM To: 'Spencer Shepler'; Van Belleghem,Audrey; bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey, Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com; andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM; trond.myklebust@fys.uio.no; Noveck, Dave; Khan, Saadia; Erasani, Pranoop Subject: RE: Kickoff: Exchange ID and destroy Client ID Here are the lines of the spec, http://www.nfsv4-editor.org/draft-13/draft-ietf-nfsv4-minorversion1-13-l n.txt that should be inspected (or simply read as noted below). Definitions 815-849 866-844 Client and Server Owners 1193-1521 Trunking 2263-2424 - Previously inspected. Will not be re-inspected but useful for background material. State Protection Model 3315-3722 - Previously inspected. Will not be re-inspected but useful for background material. EXCHANGE_ID 22726-23661 SET_SSV 25681-25786 DESTROY_CLIENTIID 25995-26034=20 > -----Original Message----- > From: Spencer Shepler [mailto:Spencer.Shepler@Sun.COM] > Sent: Tuesday, August 14, 2007 1:25 AM > To: Van Belleghem,Audrey > Cc: bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey,=20 > Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com;=20 > andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM;=20 > trond.myklebust@fys.uio.no; Noveck, Dave; Eisler, Michael > Subject: Re: Kickoff: Exchange ID and destroy Client ID >=20 >=20 > On Jul 30, 2007, at 4:28 PM, Van Belleghem,Audrey wrote: >=20 > > Hi, > > > > Thanks to those of you who have already participated in > portions of > > the NFSv4.1 specification review. For those that don't know me, I'm > > a program manager working for Brian Pawlowski in the > CTO Office > > of Network Appliance. Brian is co-chair of the IETF's NFSv4 working > > group which has submitted an Internet draft titled "NFSv4 Minor=20 > > Version 1". > > > > The next chapters we'll be reviewing are Exchange ID and Destroy=20 > > Client ID. We'll do those together. > > > > The process will work as follows: > > > > 7/30/07- Call for reviewers (please forward this message as > > appropriate) > > > > 8/3/07 @ 11:30am Pacific Time for 30 min. Moderated concall for=20 > > editors to discuss details with reviewers 865-525-0765; access: > > 1647064 We will also be looking for volunteers for the scribe,=20 > > moderator, author, and reader roles. (Please let me know if you are > > available to attend). > > > > 8/17/07 - Responses due from reviewers with comments and=20 > > suggestions. Please copy all on your responses. > > > > 8/21/07 @ 12-2pm Pacific Time - Moderated concall for review of=20 > > comments Concall # 865-525-0765; access: 1647064 (Please let me know > > if you are available to attend). > > > > TBD - Updated document with agreed upon changes. > > > > (Spencer will forward the appropriate links for Draft 13, due out > > 8/3/07) > > > > Both meetings will use the same call-in #: > > > > (865) 525-0765; 1647064 > > Spencer will host the call. > > > > We hope you can assist us by reviewing these chapters of the=20 > > document. If you are unable to assist, please feel free to > send an > > alternate representative to our meetings. > > > > If you have any questions, please do not hesitate to contact me at=20 > > audreyv@netapp.com. > > > > Thanks everyone for your continued help and support of this revision > > and review process! >=20 > I think that everyone "here" has been involved in previous reviews. =20 > As you may have heard Mike, Dave or I say, the formal review process=20 > has been a wild success. Great comments, discussions and worthwhile=20 > and needed updates to the specification. >=20 > Having said that, we were a little short on attendance during the=20 > kickoff meeting. I hope that this was a sign of many of you being on=20 > vacation but otherwise able to help out in the review of these=20 > particular sections. :-) >=20 > Please let Audrey and I know if you will be able to assist in our=20 > review. Note that if you are unable to make the conference calls for=20 > the reviews, your review and comments are still very valuable. >=20 > Spencer >=20 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Sun Sep 09 22:59:00 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZSU-0004IR-IU; Sun, 09 Sep 2007 22:57:06 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZQn-0002Mc-M8 for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:21 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUZQm-0005tW-UM for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:21 -0400 X-IronPort-AV: E=Sophos;i="4.20,228,1186383600"; d="scan'208";a="101983529" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 09 Sep 2007 19:55:20 -0700 Received: from svlexrs01.hq.netapp.com (svlexrs01.corp.netapp.com [10.57.156.158]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l8A2tKTr006530 for ; Sun, 9 Sep 2007 19:55:20 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:55:19 -0700 Received: from exsvl03.hq.netapp.com ([10.56.8.64]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:55:19 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sun, 9 Sep 2007 19:54:38 -0700 Message-ID: <0B79955C903FA0438DDA4FFCC2BA521B0299494A@exsvl03.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part II) Thread-Index: AcfeTOk1jUuB1FA+TA6l9LGNVoZQjgAKSGjQAE6QHJAEwdImwA== From: "Erasani, Pranoop" To: X-OriginalArrivalTime: 10 as > > appropriate) > > > > 8/3/07 @ 11:30am Pacific Time for 30 min. Moderated concall for=20 > > editors to discuss details with reviewers 865-525-0765; access: > > 1647064 We will also be looking for volunteers for the scribe,=20 > > moderator, author, and reader roles. (Please let me know if you are > > available to attend). > > > > 8/17/07 - Responses due from reviewers with comments and=20 > > suggestions. Please copy all on your responses. > > > > 8/21/07 @ 12-2pm Pacific Time - Moderated concall for review of=20 > > comments Concall # 865-525-0765; access: 1647064 (Please let me know > > if you are available to attend). > > > > TBD - Updated document with agreed upon changes. > > > > (Spencer will forward the appropriate links for Draft 13, due out > > 8/3/07) > > > > Both meetings will use the same call-in #: > > > > (865) 525-0765; 1647064 > > Spencer will host the call. > > > > We hope you can assist us by reviewing these chapters of the=20 > > document. If you are unable to assist, please feel free to > send an > > alternate representative to our meetings. > > > > If you have any questions, please do not hesitate to contact me at=20 > > audreyv@netapp.com. > > > > Thanks everyone for your continued help and support of this revision > > and review process! >=20 > I think that everyone "here" has been involved in previous reviews. =20 > As you may have heard Mike, Dave or I say, the formal review process=20 > has been a wild success. Great comments, discussions and worthwhile=20 > and needed updates to the specification. >=20 > Having said that, we were a little short on attendance during the=20 > kickoff meeting. I hope that this was a sign of many of you being on=20 > vacation but otherwise able to help out in the review of these=20 > particular sections. :-) >=20 > Please let Audrey and I know if you will be able to assist in our=20 > review. Note that if you are unable to make the conference calls for=20 > the reviews, your review and comments are still very valuable. >=20 > Spencer >=20 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From nfsv4-bounces@ietf.org Sun Sep 09 22:59:00 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZSU-0004IR-IU; Sun, 09 Sep 2007 22:57:06 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUZQn-0002Mc-M8 for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:21 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUZQm-0005tW-UM for nfsv4@ietf.org; Sun, 09 Sep 2007 22:55:21 -0400 X-IronPort-AV: E=Sophos;i="4.20,228,1186383600"; d="scan'208";a="101983529" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 09 Sep 2007 19:55:20 -0700 Received: from svlexrs01.hq.netapp.com (svlexrs01.corp.netapp.com [10.57.156.158]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l8A2tKTr006530 for ; Sun, 9 Sep 2007 19:55:20 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:55:19 -0700 Received: from exsvl03.hq.netapp.com ([10.56.8.64]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Sun, 9 Sep 2007 19:55:19 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sun, 9 Sep 2007 19:54:38 -0700 Message-ID: <0B79955C903FA0438DDA4FFCC2BA521B0299494A@exsvl03.hq.netapp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part II) Thread-Index: AcfeTOk1jUuB1FA+TA6l9LGNVoZQjgAKSGjQAE6QHJAEwdImwA== From: "Erasani, Pranoop" To: X-OriginalArrivalTime: 10 Sep 2007 02:55:19.0359 (UTC) FILETIME=[05CFCCF0:01C7F356] X-Spam-Score: 0.0 (/) X-Scan-Signature: 0770535483960d190d4a0d020e7060bd X-Mailman-Approved-At: Sun, 09 Sep 2007 22:57:05 -0400 Subject: [nfsv4] NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part II) X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org Please find below the review comments captured from the second review session scheduled for inspection of "Exchange ID and Destroy Client ID" related chapters. #Date# August 30, 2007, 11 AM - 2 PM PST #Participants# Mike Eisler - Network Appliance (Moderator/Author) Dave Noveck - Sun Microsystems (Inspector) Ricardo Labiaga - Network Appliance (Reader) Saadia Khan - Network Appliance (Inspector) Pranoop Erasani - Network Appliance (Scribe/Inspector) Trond Myklebust - Network Appliance (Insepctor) Robert gordon - Sun Microsystems (Inspector) Benny Halevy - Panasas (Inspector) Fred Isaman - CITI (Inspector) #Review Comments# Line No. Raised By Comment -------- --------- ------- 23370 Khan,Erasani Use SHOULD instead of MUST or current MAY Labiaga 23399-23340 Labiaga,Khan Probably there is another better place to describe the note. Noveck 23434 Noveck Can EXCHANGEID be issued along with with SEQUENCE op? Has to be explained better. Maybe described elsewhere? If so, refer that here. 23488 Eisler Case 3: No state and no lease expired, it should not return CLID_INUSE. 23506 Trond Verifier matches and we have SSV protections, what happens when the retry does not have SSV? (Suggestion: REJECTED) 23483 Khan Can Response to EXCHANGEID could be different (although CLIENTID be the same), when minor id be different? 23511 Fred (line 23164 referred) It would be nice to explain when clientid can be reused or when it is not. 23165 Fred It is referring to subsequent ExchangeID operation on a confirmed clientid 23511 Fred 4 & 5 should be made consistent. Discussion lead to consensus on changing line 23548 clientid_ret -> old_clientid_ret. 22972 Fred Is this accurate? These properties may be updated by subsequent EXCHANGEID request on a confirmed clientid. Right? 23548 Fred 4 & 5 should be merged. 23611 Khan s/clientid_ret/old_clientid_ret 23617 Khan Original clientid_ret -> new clientid_ret. 23620 Labiaga Do we need to check the principal? (Suggestion: Need not) 23661 Eisler Expand 10 to include verifier mismatch. Also, don't check principal if there is a verifier mismatch. General Eisler A server can maintain maximum of 1 confirmed and 1 unconfirmed record for each client owner. General Khan Same scenarios (1 - 10) are present in CREATE_SESSION and need to be updated. 25735 Khan what's the value of SSV the first time around. (Suggestion: Zero). Seems to have been referred at 3404. 25740 Trond Preferred algorithm for generating SSV should be more of a pseudo-random number generator. That suggestion should be mentioned here or whereever appropriate. 25753 Labiaga Revisit Change of SSV wording. What this really means? Additionally, Eisler agreed to review BIND_CONN_TO_SESSION. 23768 Labiaga Add that "ssa_ssv" not be zero. 25775 Labiaga Clarify this paragraph. Say that it's a bad idea to use SET_SSV with SSV GSS. Refer to core infrastructure chapter. 25783 Eisler/Labiaga This needs to be moved to core infrastructure or description section. This is not an implementation detail but part of the specification. 25783 Eisler/Labiaga Is internal counter an implementation detail? How is it realted to ssa_sequence? 26031 Eisler s/unsued/unused. That's all for the c Sep 2007 02:55:19.0359 (UTC) FILETIME=[05CFCCF0:01C7F356] X-Spam-Score: 0.0 (/) X-Scan-Signature: 0770535483960d190d4a0d020e7060bd X-Mailman-Approved-At: Sun, 09 Sep 2007 22:57:05 -0400 Subject: [nfsv4] NFS4.1 Spec Review: Exchange ID and destroy Client ID (Part II) X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org Please find below the review comments captured from the second review session scheduled for inspection of "Exchange ID and Destroy Client ID" related chapters. #Date# August 30, 2007, 11 AM - 2 PM PST #Participants# Mike Eisler - Network Appliance (Moderator/Author) Dave Noveck - Sun Microsystems (Inspector) Ricardo Labiaga - Network Appliance (Reader) Saadia Khan - Network Appliance (Inspector) Pranoop Erasani - Network Appliance (Scribe/Inspector) Trond Myklebust - Network Appliance (Insepctor) Robert gordon - Sun Microsystems (Inspector) Benny Halevy - Panasas (Inspector) Fred Isaman - CITI (Inspector) #Review Comments# Line No. Raised By Comment -------- --------- ------- 23370 Khan,Erasani Use SHOULD instead of MUST or current MAY Labiaga 23399-23340 Labiaga,Khan Probably there is another better place to describe the note. Noveck 23434 Noveck Can EXCHANGEID be issued along with with SEQUENCE op? Has to be explained better. Maybe described elsewhere? If so, refer that here. 23488 Eisler Case 3: No state and no lease expired, it should not return CLID_INUSE. 23506 Trond Verifier matches and we have SSV protections, what happens when the retry does not have SSV? (Suggestion: REJECTED) 23483 Khan Can Response to EXCHANGEID could be different (although CLIENTID be the same), when minor id be different? 23511 Fred (line 23164 referred) It would be nice to explain when clientid can be reused or when it is not. 23165 Fred It is referring to subsequent ExchangeID operation on a confirmed clientid 23511 Fred 4 & 5 should be made consistent. Discussion lead to consensus on changing line 23548 clientid_ret -> old_clientid_ret. 22972 Fred Is this accurate? These properties may be updated by subsequent EXCHANGEID request on a confirmed clientid. Right? 23548 Fred 4 & 5 should be merged. 23611 Khan s/clientid_ret/old_clientid_ret 23617 Khan Original clientid_ret -> new clientid_ret. 23620 Labiaga Do we need to check the principal? (Suggestion: Need not) 23661 Eisler Expand 10 to include verifier mismatch. Also, don't check principal if there is a verifier mismatch. General Eisler A server can maintain maximum of 1 confirmed and 1 unconfirmed record for each client owner. General Khan Same scenarios (1 - 10) are present in CREATE_SESSION and need to be updated. 25735 Khan what's the value of SSV the first time around. (Suggestion: Zero). Seems to have been referred at 3404. 25740 Trond Preferred algorithm for generating SSV should be more of a pseudo-random number generator. That suggestion should be mentioned here or whereever appropriate. 25753 Labiaga Revisit Change of SSV wording. What this really means? Additionally, Eisler agreed to review BIND_CONN_TO_SESSION. 23768 Labiaga Add that "ssa_ssv" not be zero. 25775 Labiaga Clarify this paragraph. Say that it's a bad idea to use SET_SSV with SSV GSS. Refer to core infrastructure chapter. 25783 Eisler/Labiaga This needs to be moved to core infrastructure or description section. This is not an implementation detail but part of the specification. 25783 Eisler/Labiaga Is internal counter an implementation detail? How is it realted to ssa_sequence? 26031 Eisler s/unsued/unused. That's all for the comments capture in the meeting. This session finished the inspection of Exchange ID and Destroy ClientID. Apologies in advance, if I have associated a wrong name with the comment. Thanks, - Pranoop -----Original Message----- From: Eisler, Michael Sent: Tuesday, August 14, 2007 6:25 AM To: 'Spencer Shepler'; Van Belleghem,Audrey; bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey, Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com; andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM; trond.myklebust@fys.uio.no; Noveck, Dave; Khan, Saadia Subject: RE: Kickoff: Exchange ID and destroy Client ID Here are the lines of the spec, http://www.nfsv4-editor.org/draft-13/draft-ietf-nfsv4-minorversion1-13-l n.txt that should be inspected (or simply read as noted below). Definitions 815-849 866-844 Client and Server Owners 1193-1521 Trunking 2263-2424 - Previously inspected. Will not be re-inspected but useful for background material. State Protection Model 3315-3722 - Previously inspected. Will not be re-inspected but useful for background material. EXCHANGE_ID 22726-23661 SET_SSV 25681-25786 DESTROY_CLIENTIID 25995-26034=20 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 omments capture in the meeting. This session finished the inspection of Exchange ID and Destroy ClientID. Apologies in advance, if I have associated a wrong name with the comment. Thanks, - Pranoop -----Original Message----- From: Eisler, Michael Sent: Tuesday, August 14, 2007 6:25 AM To: 'Spencer Shepler'; Van Belleghem,Audrey; bhalevy@panasas.com; rick@snowhite.cis.uoguelph.ca; Talpey, Thomas; Labiaga, Ricardo; Peter.Varga@hummingbird.com; andros@citi.umich.edu; iisaman@citi.umich.edu; Robert.Gordon@Sun.COM; trond.myklebust@fys.uio.no; Noveck, Dave; Khan, Saadia Subject: RE: Kickoff: Exchange ID and destroy Client ID Here are the lines of the spec, http://www.nfsv4-editor.org/draft-13/draft-ietf-nfsv4-minorversion1-13-l n.txt that should be inspected (or simply read as noted below). Definitions 815-849 866-844 Client and Server Owners 1193-1521 Trunking 2263-2424 - Previously inspected. Will not be re-inspected but useful for background material. State Protection Model 3315-3722 - Previously inspected. Will not be re-inspected but useful for background material. EXCHANGE_ID 22726-23661 SET_SSV 25681-25786 DESTROY_CLIENTIID 25995-26034=20 _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www1.ietf.org/mailman/listinfo/nfsv4 From MelkaOwens@dadoad.com Mon Sep 10 05:21:30 2007 Return-path: Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUfST-0003OV-V7 for nfsv4-archive@lists.ietf.org; Mon, 10 Sep 2007 05:21:29 -0400 Received: from aiu127.neoplus.adsl.tpnet.pl ([83.25.228.127]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUfST-0007Vs-DP for nfsv4-archive@lists.ietf.org; Mon, 10 Sep 2007 05:21:29 -0400 Received: from m9lzdnef9w3 ([102.165.114.159] helo=m9lzdnef9w3) by aiu127.neoplus.adsl.tpnet.pl ( sendmail 8.13.3/8.13.1) with esmtpa id 1mbMLX-000JQH-mP for nfsv4-archive@lists.ietf.org; Mon, 10 Sep 2007 11:29:01 +0200 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 10 Sep 2007 11:28:43 +0200 To: nfsv4-archive@lists.ietf.org From: "Melka Owens" Subject: negawenn Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 3.4 (+++) X-Scan-Signature: 8ac499381112328dd60aea5b1ff596ea Good day nfsv4-archive 52% of women think that the size of a penis makes a difference in sexual satisfaction keenan Bouteiller http://smonter.com/ From Eoin898@humanactivespaces.de Mon Sep 10 07:20:21 2007 Return-path: Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUhJV-0004Eu-Ts for nfsv4-archive@lists.ietf.org; Mon, 10 Sep 2007 07:20:21 -0400 Received: from p50886a5b.dip.t-dialin.net ([80.136.106.91]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IUhJU-0000zO-Af for nfsv4-archive@lists.ietf.org; Mon, 10 Sep 2007 07:20:21 -0400 Received: by 10.111.208.228 with SMTP id ixQRaxKhCFgHr; Mon, 10 Sep 2007 13:20:51 +0200 (GMT) Received: by 192.168.209.176 with SMTP id wDUBETfMLkTrYQ.2882085222121; Mon, 10 Sep 2007 13:20:49 +0200 (GMT) Message-ID: <000e01c7f39c$a2180780$5b6a8850@nadine1> From: "Eoin Kudrick" To: Subject: verspand Date: Mon, 10 Sep 2007 13:20:46 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C7F3AD.65A0D780" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 3.5 (+++) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 ------=_NextPart_000_0005_01C7F3AD.65A0D780 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable http://spappy.net/ Hi bro Kudrick I had always been the butt of locker room jokes about my small penis asdf Cennon ------=_NextPart_000_0005_01C7F3AD.65A0D780 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

http://spappy.net/

Hi bro Kudrick

I had always been the butt of locker room = jokes about my=20 small penis

asdf Cennon

------=_NextPart_000_0005_01C7F3AD.65A0D780-- From nfsv4-bounces@ietf.org Mon Sep 10 14:18:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUnoK-00067Y-Jb; Mon, 10 Sep 2007 14:16:36 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IUnoI-00064i-Ow for nfsv4@ietf.org; Mon, 10 Sep 2007 14:16:34 -0400 Received: from mx2.netapp.com ([216.240.18.37]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IUnoI-00077Y-5D for nfsv4@ietf.org; Mon, 10 Sep 2007 14:16:34 -0400 X-IronPort-AV: E=Sophos;i="4.20,232,1186383600"; d="scan'208";a="102251468" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 10 Sep 2007 11:16:33 -0700 Received: from svlexrs02.hq.netapp.com (svlexrs02.corp.netapp.com [10.57.156.154]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id l8AIGXb7001532 for ; Mon, 10 Sep 2007 11:16:33 -0700 (PDT) Received: from exsvlrb01.hq.netapp.com ([10.56.8.62]) by svlexrs02.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 10 Sep 2007 11:16:33 -0700 Received: from exnane01.hq.netapp.com ([10.97.0.61]) by exsvlrb01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 10 Sep 2007 11:16:32 -0700 Received: from tmt.netapp.com ([10.30.24.114]) by exnane01.hq.netapp.com with Microsoft SMTPSVC(6.0.3790.0); Mon, 10 Sep 2007 14:16:31 -0400 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 10 Sep 2007 14:15:52 -0400 To: nfsv4@ietf.org From: "Talpey, Thomas" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Message-ID: X-OriginalArrivalTime: 10 Sep 2007 18:16:31.0787 (UTC) FILETIME=[B6BEFFB0:01C7F3D6] X-Spam-Score: 0.0 (/) X-Scan-Signature: 34d35111647d654d033d58d318c0d21a Subject: [nfsv4] Fwd: [NFS] [PATCH 00/19] NFS/RDMA client support X-BeenThere: nfsv4@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: NFSv4 Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nfsv4-bounces@ietf.org FYI, the latest version of Linux client NFS/RDMA support is out. Tom. > ---------- Forwarded Message ---------- >Date: Mon, 10 Sep 2007 13:41:55 -0400 >To: nfs@lists.sourceforge.net >From: "Talpey, Thomas"