Document: draft-ietf-netmod-yang-instance-file-format-04.txt Reviewer: Acee Lindem Review Date: Oct 30st, 2019 Review Type: Working Group Last Call Intended Status: Standards Track Summary: Ready with Issues Modules: "ietf-yang-instance-data@2019-07-04.yang" Tech Summary: The model describes mechanisms and statically specifying instance data (XML or JSON) for YANG models. Use cases are also discussed although not in normative text. The document is relatively straight forward but could benefit from some editorial cleanup. Major Comments: None Minor Comments: 1. The "Security Considerations" in section 8 do not conform to the recommended template in https://trac.ietf.org/trac/ops/wiki/yang-security- guidelines>. The considerations may be completely dependent on the included instance Data Set or some of the information in the model may also be sensitive. However, it needs to be better described. 2. I feel it would be helpful to explicitly state that the both read-only and read-write instance data may be included in the instance data set. 3. The document could requires some editorial cleanup. For example, use complete sentenses for principles in section 2.1 and punctuate. Do not begin sentenses with "E.g. ...". Nits: See diff below. *** draft-ietf-netmod-yang-instance-file-format-04.txt.orig 2019-10-29 16:36:22.000000000 -0400 --- draft-ietf-netmod-yang-instance-file-format-04.txt 2019-10-29 21:40:06.000000000 -0400 *************** *** 20,26 **** running server available. This document specifies a standard file format for YANG instance data (which follows the syntax and semantic from existing YANG models, re-using the same format as the reply to a ! operation/request) and decorates it with metadata. Status of This Memo --- 20,26 ---- running server available. This document specifies a standard file format for YANG instance data (which follows the syntax and semantic from existing YANG models, re-using the same format as the reply to a ! operation/request) and annotates it with metadata. Status of This Memo *************** *** 114,127 **** Internet-Draft YANG Instance Data August 2019 ! Instance Data Set: A named set of data items decorated with metadata that can be used as instance data in a YANG data tree. Instance Data File: A file containing an instance data set formatted according to the rules described in this document. ! Content-schema: A set of YANG modules with their revision,suupported ! features and deviations for which the instance data set contains instance data Content defining Yang module(s): YANG module(s) that make up the --- 114,127 ---- Internet-Draft YANG Instance Data August 2019 ! Instance Data Set: A named set of data items annotated with metadata that can be used as instance data in a YANG data tree. Instance Data File: A file containing an instance data set formatted according to the rules described in this document. ! Content-schema: A set of YANG modules with their revision, supported ! features, and deviations for which the instance data set contains instance data Content defining Yang module(s): YANG module(s) that make up the *************** *** 138,145 **** There is a need to document data defined in YANG models when a live server is not available. Data is often needed already at design or implementation time or needed by groups that do not have a live ! running server available. To facilitate this off-line delivery of ! data this document specifies a standard format for YANG instance data sets and YANG instance data files. The following is a list of already implemented and potential use --- 138,145 ---- There is a need to document data defined in YANG models when a live server is not available. Data is often needed already at design or implementation time or needed by groups that do not have a live ! running server available. To facilitate this offline delivery of ! data, this document specifies a standard format for YANG instance data sets and YANG instance data files. The following is a list of already implemented and potential use *************** *** 153,159 **** UC4 Instance data used as backup ! UC5 Storing the configuration of a device, e.g. for archive or audit purposes UC6 Storing diagnostics data --- 153,159 ---- UC4 Instance data used as backup ! UC5 Storing the configuration of a device, e.g., for archive or audit purposes UC6 Storing diagnostics data *************** *** 186,201 **** The following is a list of the basic principles of the instance data format: ! P1 Two standard formats are based on the XML and the JSON encoding ! P2 Re-use existing formats similar to the response to a operation/request P3 Add metadata about the instance data set (Section 3, Paragraph 9) P4 A YANG instance data set may contain data for many YANG modules ! P5 Instance data may include configuration data, state data or a mix of the two P6 Partial data sets are allowed --- 186,201 ---- The following is a list of the basic principles of the instance data format: ! P1 Two standard formats are based on the XML and JSON encodings ! P2 Reuse existing formats similar to the response to a operation/request P3 Add metadata about the instance data set (Section 3, Paragraph 9) P4 A YANG instance data set may contain data for many YANG modules ! P5 Instance data may include configuration data, state data, or a mix of the two P6 Partial data sets are allowed *************** *** 227,233 **** Two formats are specified based on the XML and JSON YANG encodings. ! Later as other YANG encodings (e.g. CBOR) are defined further instance data formats may be specified. The content-data part SHALL follow the encoding rules defined in --- 227,233 ---- Two formats are specified based on the XML and JSON YANG encodings. ! Later as other YANG encodings (e.g., CBOR) are defined, further instance data formats may be specified. The content-data part SHALL follow the encoding rules defined in *************** *** 245,251 **** ignored by users of YANG instance data, allowing it to be used later for other purposes. ! in the XML format implementation specific XML attributes. Unknown attributes MUST be ignored by users of YANG instance data, allowing them to be used later for other purposes. --- 245,251 ---- ignored by users of YANG instance data, allowing it to be used later for other purposes. ! in the XML format implementation specific XML attributes, unknown attributes MUST be ignored by users of YANG instance data, allowing them to be used later for other purposes. *************** *** 271,277 **** * instance-data-set-name ['@' revision-date] '.filetype' ! * E.g. acme-router-modules@2018-01-25.xml --- 271,277 ---- * instance-data-set-name ['@' revision-date] '.filetype' ! * E.g., acme-router-modules@2018-01-25.xml *************** *** 282,288 **** Internet-Draft YANG Instance Data August 2019 ! If the leaf name is present in the instance data header this MUST be used. Revision-date MUST be set to the latest revision date inside the instance data set. --- 282,288 ---- Internet-Draft YANG Instance Data August 2019 ! If the leaf name is present in the instance data header, this MUST be used. Revision-date MUST be set to the latest revision date inside the instance data set. *************** *** 290,301 **** * instance-data-set-name ['@' timestamp] '.filetype' ! * E.g. acme-router-modules@2018-01-25T15_06_34_3+01_00.json ! If the leaf name is present in the instance data header this MUST be used. If the leaf timestamp is present in the instance data ! header this MUST be used; the semicolons and the decimal point if ! present shall be replaced by underscores. The revision date or timestamp is optional. ".filetype" SHALL be ".json" or ".xml" according to the format used. --- 290,301 ---- * instance-data-set-name ['@' timestamp] '.filetype' ! * E.g,. acme-router-modules@2018-01-25T15_06_34_3+01_00.json ! If the leaf name is present in the instance data header, this MUST be used. If the leaf timestamp is present in the instance data ! header, this MUST be used; the semicolons and the decimal point, if ! present shall, be replaced by underscores. The revision date or timestamp is optional. ".filetype" SHALL be ".json" or ".xml" according to the format used. *************** *** 315,321 **** 3.1. Specifying the Content Schema ! To properly understand and use an instance data set the user needs to know the content-schema. One of the following methods SHOULD be used: --- 315,321 ---- 3.1. Specifying the Content Schema ! To properly understand and use an instance data set, the user needs to know the content-schema. One of the following methods SHOULD be used: *************** *** 342,359 **** already known, or the information is available through external documents. ! Additional methods e.g. a YANG-package based solution may be added later. Note, the specified content-schema only indicates the set of modules that were used to define this YANG instance data set. Sometimes instance data may be used for a server supporting a different YANG ! module set. (e.g. for "UC2 Preloading Data" the instance data set may not be updated every time the YANG modules on the server are updated) Whether the instance data set is usable for a possibly different real-life YANG module set depends on many factors including the ! compatibility between the specified and the real-life YANG module set ! (considering modules, revisions, features, deviations), the scope of the instance data, etc. 3.1.1. Inline Method --- 342,359 ---- already known, or the information is available through external documents. ! Additional methods, e.g., a YANG-package based solution may be added later. Note, the specified content-schema only indicates the set of modules that were used to define this YANG instance data set. Sometimes instance data may be used for a server supporting a different YANG ! module set. (e.g., for "UC2 Preloading Data" the instance data set may not be updated every time the YANG modules on the server are updated) Whether the instance data set is usable for a possibly different real-life YANG module set depends on many factors including the ! compatibility between the specified and the real-life YANG module set, ! considering modules, revisions, features, deviations, the scope of the instance data, etc. 3.1.1. Inline Method *************** *** 361,372 **** One or more inline-target-spec elements define YANG module(s) used to specify the content defining YANG modules. ! E.g. ietf-yang-library@2016-06-21.yang The anydata inline-content-schema carries instance data (conforming to the inline-target-spec modules) that actually specifies the content defining YANG modules including revision, supported features, ! deviations and any relevant additional data (e.g. version labels) 3.1.2. Simplified-Inline Method --- 361,372 ---- One or more inline-target-spec elements define YANG module(s) used to specify the content defining YANG modules. ! E.g., ietf-yang-library@2016-06-21.yang The anydata inline-content-schema carries instance data (conforming to the inline-target-spec modules) that actually specifies the content defining YANG modules including revision, supported features, ! deviations and any relevant additional data (e.g., version labels) 3.1.2. Simplified-Inline Method *************** *** 384,390 **** The referenced instance data file MAY have no content-data if it is used solely for specifying the content-schema. The referenced YANG instance data file might use the INLINE method or might use the URI ! method to reference further instance data file(s). However at the --- 384,390 ---- The referenced instance data file MAY have no content-data if it is used solely for specifying the content-schema. The referenced YANG instance data file might use the INLINE method or might use the URI ! method to reference further instance data file(s). However, at the *************** *** 397,416 **** end of this reference chain there MUST be an instance data file using the INLINE method. ! If a referenced instance data file is not available the revision ! data, supported features and deviations for the target YANG modules are unknown. The URI method is advantageous when the user wants to avoid the overhead of specifying the content-schema in each instance data file: ! E.g. In Use Case 6, when the system creates a diagnostic file every minute to document the state of the server. 3.2. Examples The following example is based on "UC1, Documenting Server Capabilities". It provides (a shortened) list of supported YANG ! modules and Netconf capabilities for a server. It uses the inline method to specify the content-schema. --- 397,416 ---- end of this reference chain there MUST be an instance data file using the INLINE method. ! If a referenced instance data file is not available, the revision ! data, supported features, and deviations for the target YANG modules are unknown. The URI method is advantageous when the user wants to avoid the overhead of specifying the content-schema in each instance data file: ! E.g., in Use Case 6, when the system creates a diagnostic file every minute to document the state of the server. 3.2. Examples The following example is based on "UC1, Documenting Server Capabilities". It provides (a shortened) list of supported YANG ! modules and NETCCONF capabilities for a server. It uses the inline method to specify the content-schema. *************** *** 624,630 **** "schema-uri": "file:///acme-netconf-diagnostics-yanglib.json", "timestamp": "2018-01-25T17:00:38Z", "description": ! "Netconf statistics", "content-data": { "ietf-netconf-monitoring:netconf-state": { "statistics": { --- 624,630 ---- "schema-uri": "file:///acme-netconf-diagnostics-yanglib.json", "timestamp": "2018-01-25T17:00:38Z", "description": ! "NETCONF statistics", "content-data": { "ietf-netconf-monitoring:netconf-state": { "statistics": { *************** *** 647,661 **** 4. Data Life cycle ! In UC2 "Preloading default configuration data" the loaded data may be ! changed later e.g. by management operations. In UC6 "Storing ! Diagnostics data" the diagnostics values may change on device every second. ! YANG instance data is a snap-shot of information at a specific point ! of time. If the data changes afterwards this is not represented in ! the instance data set anymore. The valid values can be retrieved in ! run-time via NETCONF/RESTCONF or received e.g. in Yang-Push notifications. Whether the instance data changes and if so, when and how, SHOULD be --- 647,661 ---- 4. Data Life cycle ! In UC2 "Preloading default configuration data", the loaded data may be ! changed later, e.g., by management operations. In UC6 "Storing ! Diagnostics data", the diagnostics values may change on the device every second. ! YANG instance data is a snapshot of information at a specific point ! of time. If the data changes afterwards, this is not represented in ! the instance data set anymore. The valid values can be retrieved at ! run-time via NETCONF/RESTCONF or received, e.g., in YANG-Push notifications. Whether the instance data changes and if so, when and how, SHOULD be *************** *** 678,688 **** Instance data sets that are produced as a result of some sort of specification or design effort SHOULD be available without the need ! for a live server e.g. via download from the vendor's website, or in ! any other way product documentation is distributed. Other instance data sets may be read from or produced by the YANG ! server itself e.g. UC6 documenting diagnostic data. 6. Backwards Compatibility --- 678,688 ---- Instance data sets that are produced as a result of some sort of specification or design effort SHOULD be available without the need ! for a live server, e.g., via download from the vendor's website, or in ! any other way that product documentation is distributed. Other instance data sets may be read from or produced by the YANG ! server itself, e.g., UC6 documenting diagnostic data. 6. Backwards Compatibility *************** *** 691,714 **** dependent on the specific use case and the content-schema. For instance data that is the result of a design or specification ! activity some changes that may be good to avoid are listed. YANG uses the concept of managed entities identified by key values; if the connection between the represented entity and the key value is not ! preserved during an update this may lead to problems. o If the key value of a list entry that represents the same managed entity as before is changed, the user may mistakenly identify the list entry as new. o If the meaning of a list entry is changed, but the key values are ! not (e.g. redefining an alarm-type but not changing its alarm- type-id) the change may not be noticed. o If the key value of a previously removed list entry is reused for ! a different entity, the change may be mis-interpreted as reintroducing the previous entity. ! 7. Yang Instance Data Model 7.1. Tree Diagram --- 691,714 ---- dependent on the specific use case and the content-schema. For instance data that is the result of a design or specification ! activity, some changes that may be good to avoid are listed. YANG uses the concept of managed entities identified by key values; if the connection between the represented entity and the key value is not ! preserved during an update, this may lead to problems. o If the key value of a list entry that represents the same managed entity as before is changed, the user may mistakenly identify the list entry as new. o If the meaning of a list entry is changed, but the key values are ! not (e.g., redefining an alarm-type but not changing its alarm- type-id) the change may not be noticed. o If the key value of a previously removed list entry is reused for ! a different entity, the change may be misinterpreted as reintroducing the previous entity. ! 7. YANG Instance Data Model 7.1. Tree Diagram *************** *** 812,818 **** sx:structure instance-data-set { description "A data structure to define a format for a ! YANG instance data set.Consists of meta-data about the instance data set and the real content-data."; leaf name { --- 812,818 ---- sx:structure instance-data-set { description "A data structure to define a format for a ! YANG instance data sets. Consists of meta-data about the instance data set and the real content-data."; leaf name { *************** *** 851,866 **** min-elements 1; ordered-by user; description ! "Indicates that content defining Yang modules ! are specified inline. Each value MUST be a YANG Module name including the revision-date as defined for YANG file names in RFC7950. ! E.g. ietf-yang-library@2016-06-21.yang The first item is either ietf-yang-library or some other YANG module that contains a list of YANG modules with ! their name, revision-date, supported-features and deviations. As some versions of ietf-yang-library MAY contain different module-sets for different datastores, if --- 851,866 ---- min-elements 1; ordered-by user; description ! "Indicates that content defining YANG modules ! are specified inline. Each value MUST be a YANG Module name including the revision-date as defined for YANG file names in RFC7950. ! E.g., ietf-yang-library@2016-06-21.yang The first item is either ietf-yang-library or some other YANG module that contains a list of YANG modules with ! their name, revision-date, supported-features, and deviations. As some versions of ietf-yang-library MAY contain different module-sets for different datastores, if *************** *** 871,883 **** datastore. Subsequent items MAY specify YANG modules augmenting the ! first module with useful data (e.g. a version label)."; } anydata inline-content-schema { mandatory true; description "Instance data corresponding to the YANG modules specified in the inline-spec nodes defining the set ! of content defining Yang YANG modules for this instance-data-set."; } } --- 871,883 ---- datastore. Subsequent items MAY specify YANG modules augmenting the ! first module with useful data (e.g., a version label)."; } anydata inline-content-schema { mandatory true; description "Instance data corresponding to the YANG modules specified in the inline-spec nodes defining the set ! of content defining YANG modules for this instance-data-set."; } } *************** *** 888,894 **** description "A reference to another YANG instance data file. This instance data file will use the same set of target ! YANG modules, revisions, supported features and deviations as the referenced YANG instance data file."; --- 888,894 ---- description "A reference to another YANG instance data file. This instance data file will use the same set of target ! YANG modules, revisions, supported features, and deviations as the referenced YANG instance data file."; *************** *** 923,932 **** leaf datastore { type ds:datastore-ref; description "The identity of the datastore with which the ! instance data set is associated e.g. the datastore from ! where the data was read or the datastore where the data could be loaded or the datastore which is being documented. ! If a single specific datastore can not be specified, the leaf MUST be absent. If this leaf is absent, then the datastore to which the --- 923,932 ---- leaf datastore { type ds:datastore-ref; description "The identity of the datastore with which the ! instance data set is associated, e.g., the datastore from ! where the data was read or the datastore from which the data could be loaded or the datastore which is being documented. ! If a single specific datastore cannot be specified, the leaf MUST be absent. If this leaf is absent, then the datastore to which the *************** *** 1222,1228 **** A server has a number of server-capabilities that are defined in YANG modules and can be retrieved from the server using protocols like ! NETCONF or RESTCONF. server capabilities include --- 1222,1228 ---- A server has a number of server-capabilities that are defined in YANG modules and can be retrieved from the server using protocols like ! NETCONF or RESTCONF. Server capabilities include: *************** *** 1235,1246 **** o data defined in ietf-yang-library: YANG modules, submodules, ! features, deviations, schema-mounts, datastores supported ([I-D.ietf-netconf-rfc7895bis]) o alarms supported ([I-D.ietf-ccamp-alarm-module]) ! o data nodes, subtrees that support or do not support on-change notifications ([I-D.ietf-netconf-yang-push]) o netconf-capabilities in ietf-netconf-monitoring --- 1235,1246 ---- o data defined in ietf-yang-library: YANG modules, submodules, ! features, deviations, schema-mounts, and datastores supported ([I-D.ietf-netconf-rfc7895bis]) o alarms supported ([I-D.ietf-ccamp-alarm-module]) ! o data nodes and subtrees that support or do not support on-change notifications ([I-D.ietf-netconf-yang-push]) o netconf-capabilities in ietf-netconf-monitoring *************** *** 1248,1280 **** While it is good practice to allow a client to query these capabilities from the live server, that is often not possible. ! Often when a network node is released an associated NMS (network management system) is also released with it. The NMS depends on the ! capabilities of the server. During NMS implementation information about server capabilities is needed. If the information is not ! available early in some off-line document, but only as instance data from the live network node, the NMS implementation will be delayed, ! because it has to wait for the network node to be ready. Also assuming that all NMS implementors will have a correctly configured ! network node available to retrieve data from, is a very expensive proposition. (An NMS may handle dozens of node types.) Network operators often build their own home-grown NMS systems that ! needs to be integrated with a vendor's network node. The operator needs to know the network node's server capabilities in order to do ! this. Moreover the network operator's decision to buy a vendor's product may even be influenced by the network node's OAM feature set ! documented as the Server's capabilities. Beside NMS implementors, system integrators and many others also need the same information early. Examples could be model driven testing, generating documentation, etc. Most server-capabilities are relatively stable and change only during ! upgrade or due to licensing or addition or removal of HW. They are ! usually defined by a vendor at design time, before the product is ! released. It feasible and advantageous to define/document them early ! e.g. in a YANG instance data File. It is anticipated that a separate IETF document will define in detail how and which set of server capabilities should be documented. --- 1248,1280 ---- While it is good practice to allow a client to query these capabilities from the live server, that is often not possible. ! Often when a network node is released, an associated NMS (network management system) is also released with it. The NMS depends on the ! capabilities of the server. During NMS implementation, information about server capabilities is needed. If the information is not ! available early in some offline document, but only as instance data from the live network node, the NMS implementation will be delayed, ! because it has to wait until the network node is ready. Also assuming that all NMS implementors will have a correctly configured ! network nodes from whic data is to be retrieved, is a very expensive proposition. (An NMS may handle dozens of node types.) Network operators often build their own home-grown NMS systems that ! need to be integrated with a vendor's network node. The operator needs to know the network node's server capabilities in order to do ! this. Moreover, the network operator's decision to buy a vendor's product may even be influenced by the network node's OAM feature set ! documented as the server's capabilities. Beside NMS implementors, system integrators and many others also need the same information early. Examples could be model driven testing, generating documentation, etc. Most server-capabilities are relatively stable and change only during ! upgrade or due to licensing or the addition or removal of hardware. They ! are usually defined by a vendor at design time, before the product is ! released. It feasible and advantageous to define/document them early, ! e.g., in a YANG instance data File. It is anticipated that a separate IETF document will define in detail how and which set of server capabilities should be documented. *************** *** 1293,1310 **** C.1.2. Use Case 2: Preloading Data There are parts of the configuration that must be fully configurable ! by the operator, however for which often a simple default ! configuration will be sufficient. One example is access control groups/roles and related rules. While ! a sophisticated operator may define dozens of different groups often a basic (read-only operator, read-write system administrator, security-administrator) triplet will be enough. Vendors will often provide such default configuration data to make device configuration easier for an operator. ! Defining Access control data is a complex task. To help the device ! vendor pre-defines a set of default groups (/nacm:nacm/groups) and rules for these groups to access specific parts of common models (/nacm:nacm/rule-list/rule). --- 1293,1310 ---- C.1.2. Use Case 2: Preloading Data There are parts of the configuration that must be fully configurable ! by the operator. However, often a simple default configuration will ! be sufficient. One example is access control groups/roles and related rules. While ! a sophisticated operator may define dozens of different groups, often a basic (read-only operator, read-write system administrator, security-administrator) triplet will be enough. Vendors will often provide such default configuration data to make device configuration easier for an operator. ! Defining access control data is a complex task. To help, the device ! vendor predefines a set of default groups (/nacm:nacm/groups) and rules for these groups to access specific parts of common models (/nacm:nacm/rule-list/rule). *************** *** 1315,1330 **** Nearly every server has a factory default configuration. If the system is really badly misconfigured or if the current configuration ! is to be abandoned the system can be reset to this default. ! In Netconf the operation can already be used to reset the startup datastore. There are ongoing efforts to introduce a new, more generic reset-datastore operation for the same purpose ! [I-D.wu-netconf-restconf-factory-restore] The operator currently has no way to know what the default ! configuration actually contains. YANG instance data can be used to ! document the factory default configuration. Authors' Addresses --- 1315,1331 ---- Nearly every server has a factory default configuration. If the system is really badly misconfigured or if the current configuration ! is to be abandoned the, system can be reset to the default factory ! configuration. ! In NETCONF, the operation can already be used to reset the startup datastore. There are ongoing efforts to introduce a new, more generic reset-datastore operation for the same purpose ! [I-D.wu-netconf-restconf-factory-restore]. The operator currently has no way to know what the default ! configuration actually contains. YANG instance data can also be used ! to document the factory default configuration. Authors' Addresses