1 jCleanCim introductionJuly 2016 jCleanCim introduction July 2016 (02v01) Copyright Tatjana (Tanja) Kostic July 2016 Copyright Tatjana (Tanja) Kostic
2 Copyright 2009-2016 Tatjana (Tanja) KosticAbout jCleanCim July 2016 jCleanCim is an open source tool: since 02v00 provided under terms of GNU LGPLv3 license Developed to support validation and documentation generation from Enterprise Architect CIM and IEC UML models. A Java application, but (for some tasks) platform dependent due to usage of applications available on MS Windows only: Enterprise Architect MS Word A console application, currently without any GUI. Use the latest version available. Copyright Tatjana (Tanja) Kostic July 2016 Copyright Tatjana (Tanja) Kostic
3 Who should use jCleanCim?Primarily those who edit CIM or IEC61850 UML and publish its documentation, thus: Official IEC CIM model editors, responsible for maintaining the CIM information model (UML) and for generating official IEC documents, and, Official IEC model managers, responsible for maintaining the IEC UML model and for generating official IEC documents, and, Official IEC CIM profile document editors, if their profiles are available in UML, for generating official IEC documents, and, Those who define custom (non-standard) CIM or IEC extensions who want to ensure they have followed standard UML modelling rules and who want to generate documentation for those extensions. Copyright Tatjana (Tanja) Kostic July 2016
4 When should you use jCleanCim?After editing CIM or IEC61850 UML, to validate the edits (reinforce rules). When you need to collect the numbers (model statistics). To produce MS Word documentation from UML models. To produce Web Access XML from UML models. If you are a CIMTool user: You would first use jCleanCim to validate correctness of the CIM information model (UML) / IEC UML model, and if required, to generate the information model documentation in MS Word or XML format, as required by IEC process. You would then use CIMTool to create CIM profiles (XSD, RDFS, OWL) and their documentation (HTML) from the imported CIM UML model, and to validate instance files created based on those profiles – independently of jCleanCim. If you are a UML-based profiling tools user: You are probably already using jCleanCim to generate MS Word documentation. Copyright Tatjana (Tanja) Kostic July 2016
5 Copyright 2009-2016 Tatjana (Tanja) KosticDocumentation Start from this presentation and the readme.html, available on-line and also bundled with every distribution. Once you unzip a jCleanCim distribution, detailed documentation is available under the project’s .doc directory: As javadoc in doc/api/index.html For binary distribution, also as .pdf (auto-generated from the javadoc) For source distribution, also build targets dependencies Copyright Tatjana (Tanja) Kostic July 2016
6 Selecting distributionbin distribution for jCleanCim end-user src distribution for jCleanCim developer, packager and end-user, with Apache ant contains also eclipse project files (unzip, then import -> existing project) Copyright Tatjana (Tanja) Kostic July 2016
7 Note for users with 64-bit Windows 7If you have a 64-bit Windows OS: ensure you install a 32-bit Java runtime (JRE) if you run a binary distribution, or Java SDK (software development kit) if you run a source distribution ensure you have that 32-bit Java appear on your PATH before potentially already installed 64-bit Java Reason: Enterprise Architect is still a 32-bit application and requires a 32-bit Java One possible fix, lasting until next reboot: See the commented text in the run.bat script in jCleanCim distribution Uncomment this line, by removing the initial “rem” This will put your 32-bit Java runtime before a potential 64-bit installation rem set PATH=C:\Program Files (x86)\Java\Jre7\bin;%PATH% Copyright Tatjana (Tanja) Kostic July 2016
8 Note for CIMTool users (1/2)In contrast to CIMTool, which is an eclipse-based application with a GUI: jCleanCim only uses eclipse for development and compilation jCleanCim is a simple, console application, without a GUI The most comfortable way to use jCleanCim is however with *-src.zip distribution in eclipse, because: You click to run jCleanCim instead of typing commands in the console window Eclipse gives a nice console output (you can copy/paste/search/scroll easily) Since 02v00, you’ll need to import not eclipse project archive, but simply an existing eclipse project (unzipped directory) If you are developing in Java with eclipse, you already have what is needed. Copyright Tatjana (Tanja) Kostic July 2016
9 Note for CIMTool users (2/2)If you have CIMTool that contains an eclipse runtime: Using that installation of eclipse (runtime) is not sufficient, because it is only runtime, without support for Java code development You must have an SDK (software development kit) to automatically build the jCleanCim application from sources On eclipse download site, the minimum required distribution is “Eclipse IDE for Java developers” you can then install CIMTool plug-in in this (or more recent) version of eclipse Copyright Tatjana (Tanja) Kostic Neon 4.6: July 2016
10 Features & configuration overviewUML model export to XMI UML model validation UML model statistics MS Word doc generation from UML (and from CIM profiles*) XML doc generation from UML CIM profiles vs. UML model cross-check* Copyright Tatjana (Tanja) Kostic *being implemented July 2016
11 jCleanCim features overview.doc / .docx .eap (UML) Current features .doc / .docx from-EA builder in-memory UML model doc collector MS Word writer .xmi (1.1) UML validators XML writer .xml (specs) .xmi (2.1) .xmi (CIMTool) UML stats reporter .xml (docs) .csv Copyright Tatjana (Tanja) Kostic .xsd .xsd from-XSD builder profiles/UML cross check .log .xsd Planned features (CIM) July 2016
12 jCleanCim features intro (1/2)jCleanCim first: Creates in-memory representation of the whole content of UML from .eap file (if set in properties – see next slide) Can export the model to the three XMI formats Can selectively export one or more XMI formats On the fly It analyses the model and calculates a bunch of things, including effective dependencies It logs that all into log files under log directory, automatically created on the first run Copyright Tatjana (Tanja) Kostic July 2016
13 jCleanCim features intro (2/2)After that, depending on what is set in properties (see next slide), one or more of the following gets executed and logged: Validation of a UML model provided in an .eap file: UML of standard IEC CIM (base and extensions), UML of IEC61850 family, and custom extensions of any of these Calculation and printing of statistics of the UML model Generation of MS Word documentation from the UML model Generation of XML documentation from the UML model (CIM only, being implemented) Generation of MS Word documentation from CIMTool .xsd profiles (CIM only, being implemented) Cross-check of profiles against the CIM UML model. Copyright Tatjana (Tanja) Kostic July 2016
14 jCleanCim configuration (1/2)Use file config/config.properties. Minimum configuration for CIM* validation and stats: Minimum configuration for automatic XMI export (used by CIM model managers): You can define a number of
15 jCleanCim configuration (2/2)Minimum configuration for CIM* MS Word doc generation: Minimum configuration for CIM* XML doc generation: model.filename = base-small.eap model.builder = sqlxml docgen.on = true profiles.docgen.on = false docgen.word.useDocFormat = true docgen.word.saveReopenEvery = 12 docgen.word.inTemplate = base-small-template.doc docgen.word.outDocument = base-small.doc ** model.filename = base-small.eap model. builder = sqlxml docgen.on = true profiles.docgen.on = false docgen.xml.outSpec = base-small-tool01v06-spec.xml docgen.xml.outDoc = base-small-tool01v06-doc.xml Copyright Tatjana (Tanja) Kostic * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class ** Preparing for faster implementation based on OpenXML (.docx) July 2016
16 Copyright 2009-2016 Tatjana (Tanja) KosticjCleanCim input files For any function, you need at least an .eap model file for MS Word document generation, you also need an MS Word template (regular .docx file) containing particular jCleanCim placeholders (CIM only, not implemented yet) for profile cross-check with the UML model, you also need one or more profiles with XSD syntax, generated by CIMTool All distributions contain sets of files in the project’s input directory: base-small.eap – small subset of base CIM and IEC with lots of buggy constructs, on purpose, to show DON’T’s sub-directories within input/profiles contain trimmed samples of CPSM and of one metering profile Copyright Tatjana (Tanja) Kostic Copy your own .eap and .doc/.docx files to the project’s input directory. (CIM only) Copy your .xsd files anywhere below the project’s input/profiles directory. July 2016
17 Recommended UML model structure (1/2)One root (here: “Model”) Currently, jCleanCim ignores everything but the first root in the .eap project Any number of model packages under the root Each with either CIM (default) or IEC61850 nature IEC61850 nature must be explicitly specified in config61850.properties file, with model.nature.iec61850 property Any number of top-level packages (per WG owner) under the model package A top-level package expected to contain a (UML) version class with correct name (example from base-small.eap, does not reflect the full model, just small part of it) Copyright Tatjana (Tanja) Kostic model.nature.iec = IEC61850Domain, My61850Extensions July 2016
18 Recommended UML model structure (2/2)Rationale Preserves the current standard CIM “place” in the .eap project Clearly separates CIM and non-CIM models Nature need not be encoded in UML, but in properties file Clearly separates standard UML and non-standard extensions Easy to evolve/update/merge standard model as it evolves Easy to independently evolve/update/merge variations of custom extensions, without interfering with the standard UML Top-level packages may be associated with IEC TC57 WGs (or to projects, for custom extensions) (example from base-small.eap, does not reflect the full model, just small part of it) Copyright Tatjana (Tanja) Kostic July 2016
19 Standard UML top-package ownersCurrently, jCleanCim encodes the mapping of top-level packages and IEC owners: CIM owners are WG13, WG14, WG16 IEC61850 owners are WG10, WG17, WG18, JWG25 (and WG19) All other UML packages and elements get assigned the owner “OTHER_CIM” or “OTHER_IEC61850”. WG14 WG13 WG16 (example from base-small.eap, does not reflect the full model, just small part of it) Copyright Tatjana (Tanja) Kostic July 2016
20 Features: Intro - model buildingOr In eternal quest for speeding up the slow EA API implementation Copyright Tatjana (Tanja) Kostic July 2016
21 Reading model from .eap repository: Minimum configuration*For any feature relying on .eap UML model, you must specify the UML model file name. Copy your own model file(s) into the project’s input directory. Since 01v08, we have 3 implementations – see next slide for comparison. model.filename = base-small.eap Default: Use if you don’t need diagram or XMI export. Useful also for doc-generation without diagrams. Use for preparing a release (XMI export) and for full document generation. Avoid !(kept as a fallback in case EA changes its internal DB schema). model.filename = base-small.eap model.builder = db OR model.filename = base-small.eap model.builder = sqlxml Copyright Tatjana (Tanja) Kostic model.filename = base-small.eap model.builder = japi * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class July 2016
22 Reading model from .eap repository: Builder comparisonsmodel.builder= db sqlxml japi how it reads .eap model file as Access DB EA Java API: queries (SQL) + result-set (XML) EA Java API: iterating content speed: iterating model as fast as it gets pretty fast extremely slow opening .eap file very slow needs ea.jar + ea.dll no yes bound to M$ Windows can export UML diagrams never yes (if docgen.on=true) can export XMI yes (if xmiexport.on=true) Copyright Tatjana (Tanja) Kostic July 2016
23 Features: UML model export to XMIUseful for CIM model managers Copyright Tatjana (Tanja) Kostic July 2016
24 XMI export: Minimum configuration*You must specify the UML model file name, a builder that uses EA repository API, and enable XMI export. Copy your own model file(s) into the project’s input directory. This will export all the supported formats: except for ‘cimtool’ - because CIMTool now (since 1.9.6) can import .eap and does not need .xmi, which is 82 MB for latest CIM ! model.filename = base-small.eap model.builder = sqlxml xmiexport.on = true Copyright Tatjana (Tanja) Kostic * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class July 2016
25 Copyright 2009-2016 Tatjana (Tanja) KosticXMI export: Overview CIM model managers need to export UML model to several XMI formats and package those .xmi files into release Manually, it's a tedious, error-prone and time-consuming process jCleanCim can do that automatically We take the model file name, and replace its .eap extension appropriately exported files go to the project‘s output directory IEC61850 model managers don’t need this functionality (yet). Copyright Tatjana (Tanja) Kostic July 2016
26 XMI export: Fine tuningThis will export only XMI appropriate for CIMTool (Rose UML 1.4, without diagrams) model.filename = base-small.eap model.builder = sqlxml xmiexport.on = true xmiexport.dialects = cimtool This will export all 3 currently supported formats (two defaults, plus ‘cimtool’) model.filename = base-small.eap model.builder = sqlxml xmiexport.on = true xmiexport.dialects = ea_xmi11, ea_xmi21, cimtool Copyright Tatjana (Tanja) Kostic July 2016
27 Features: UML model validationCompiler is my friend. Copyright Tatjana (Tanja) Kostic July 2016
28 Validation: Minimum configuration*You must specify the UML model file name and enable validation. Copy your own model file(s) into the project’s input directory. model.filename = base-small.eap validation.on = true Copyright Tatjana (Tanja) Kostic * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class July 2016
29 Validation: Overview (1/2)Validators for 7 kinds of UML elements Associations Attributes Classes Packages Diagrams Dependencies (hand-drawn in UML), and Operations (for UML of IEC61850 only) Validators deal with rules: Model consistency CIM/IEC61850 UML naming and design rules Potential model editor errors Identifying remains from Rose or other imported models Illegal/redundant UML constructs (that EA sometimes allows…) Copyright Tatjana (Tanja) Kostic July 2016
30 Validation: Overview (2/2)Each rule is a class Inheriting from a common abstract class, and implementing an interface Javadoc in package org.iec.tc57.jcleancim.validation provides guidelines on how to add a rule to a validator It is easy to add a new rule on need! Full list of rules currently available is on the next two slides, with a couple of annotated examples for class validator rules Some rules apply in general, some apply to CIM models only, and some to IEC61850 models only List is produced through log (also CIM-specific or specific) Javadoc also contains UML for all the classes (not shown below) Copyright Tatjana (Tanja) Kostic July 2016
31 Validation rules: ClassesAvailable rules in ClassValidator (category, severity): CimClassesThatShouldNotBeAbstract (modellingRule, high) CimClassesWithUnexpectedElements (permissiveTool, high) CimClassesThatShouldNotHaveOperations (modellingRule, high) ClassesWithUnexpectedConnectors (permissiveTool, high) CimClassesThatShouldNotHaveExplicitDependencies (modellingRule, high) EnumClassesWithNoLiterals (modellingRule, high) ClassesThatShouldNotHaveNestingThroughAttribute (modellingRule, high) CimCompoundClassesWithNoAttributes (modellingRule, high) Iec61850ClassesThatShouldHaveAliasAsTitle (modellingRule, high) EnumClassesWithSingleLiteral (modellingRule, medium) Iec61850ClassesThatShouldHaveTaggedValuesForDocgen (modellingRule, high) EnumClassesWithTwoLiterals (modellingRule, low) CimClassesNeverUsedInRelationships (modellingRule, high) EnumClassesWithBadName (modellingRule, high) ClassesWithUnallowedTagNames (modellingRule, high) CimPrimitiveClassesWithAttributes (modellingRule, high) Iec61850ClassesWithInvalidConstraints (modellingRule, high) CimPrimitiveClassesWithIllegalOwner (modellingRule, high) Iec61850LNClassesWithSuperfluousConstraints (modellingRule, high) ClassesWithDuplicateInheritedAttributeNames (modellingRule, high) Iec61850ClassesWithMissingCondIDTextInConstraints (modellingRule, high) ClassesWithDuplicateOwnOrInheritedAssociationEndNames (modellingRule, high) CimDatatypeClassesWithInvalidAttributes (modellingRule, high) ClassesWithSelfInheritance (permissiveTool, high) ClassesMissingDoc (documentationRule, medium) ClassesWithSelfDependency (permissiveTool, high) ClassesWithBadDocStart (documentationRule, low) ClassesWithLeafPropSet (modellingRule, high) ClassesWithBadDocEnd (documentationRule, low) ClassesWithRootPropSet (modellingRule, high) ClassesWithBadCharacterInName (namingRule, high) ClassesWithPersistentPropSet (modellingRule, high) CimClassesNameStartingWithLowerCase (namingRule, high) ClassesWithMultipleSuperclasses (modellingRule, high) CimClassesNameShouldBeSingular (namingRule, high) ClassesWithSuperclassesFromUnallowedPackage (modellingRule, high) Iec61850LNClassesInWrongGroup (namingRule, medium) ClassesThatShouldNotBeAssociationClass (modellingRule, high) Iec61850LNClassesMalformedName (namingRule, high) ClassesWithUnallowedStereotype (modellingRule, high) EnumClassesWithSomeCodesMissing (modellingRule, high) CimClassesWithOldDatatypeStereotype (modellingRule, medium) EnumClassesWithDuplicateCodes (modellingRule, high) CimClassesUsedForAttributesButHaveAssociations (modellingRule, high) ClassesWithSameName (modellingRule, high) CimClassesUsedForAttributesButHaveSubclasses (modellingRule, high) CimClassesNeverUsedAsTypeForAttribute (modellingRule, high) CimClassesUsedForAttributesButHaveSuperclasses (modellingRule, high) CIM rules Style warning Editor’s error Model inconsistencies Remains from Rose EA allows for this Copyright Tatjana (Tanja) Kostic July 2016
32 Copyright 2009-2016 Tatjana (Tanja) KosticValidation rules: Dependencies (hand-drawn), diagrams, operations, packages, associations, attributes Available rules in PackageValidator (category, severity): AttributesWithBadDocEnd (documentationRule, low) Available rules in AssociationValidator (category, severity): PackageUnexpectedElements (modellingRule, medium) CimAttributesWithBadCharacterInName (namingRule, high) AssociationsWithExplicitDirection (modellingRule, high) PackageUnexpectedConnectors (modellingRule, medium) Iec61850AttributesWithBadCharacterInName (namingRule, high) AssociationsWithRoleBadDirection (modellingRule, high) PackagesWithSelfDependency (permissiveTool, high) Iec61850DOAttributesWithTooLongName (namingRule, high) AssociationsWithDoc (documentationRule, low) PackagesWithUnallowedStereotype (modellingRule, high) Iec61850FCDAAttributesWithMissingConstraint (modellingRule, high) AssociationsWithSameDocOnBothEnds (documentationRule, medium) PackagesTopLevelWithoutVersionClass (modellingRule, high) AttributesWithInexistingEnumLiteralAsInitValue (permissiveTool, high) AssociationsWithName (namingRule, medium) Iec61850PackagesThatShouldHaveAliasAsTitle (modellingRule, high) Iec61850DOAttributesWithNameMissingAbbreviation (modellingRule, high) AssociationsWithUnallowedStereotype (modellingRule, high) PackagesWithUnallowedTagNames (modellingRule, high) CimAttributesNameStartingWithUpperCase (namingRule, high) AssociationEndsWithUnallowedStereotype (modellingRule, high) PackagesMissingDoc (documentationRule, medium) CimAttributesNameShouldBeSingular (namingRule, high) AssociationsMissingInformativeStereotype (modellingRule, high) PackagesWithBadDocStart (documentationRule, low) CimAttributesNameShouldNotStartWithClassName (namingRule, medium) AssociationsWithUnallowedTagNames (modellingRule, high) PackagesWithBadDocEnd (documentationRule, low) Iec61850AbbreviationLiteralsNameStartingWithLowerCase (namingRule, high) AssociationEndsWithUnallowedTagNames (modellingRule, high) PackagesWithBadCharacterInName (namingRule, high) Iec61850DOAttributesNameStartingWithLowerCase (namingRule, high) AssociationsWithNoMultiplicity (modellingRule, high) PackagesWithSameName (modellingRule, high) Iec61850DOAbbreviationLiteralsDuplicateName (modellingRule, high) AssociationsWithWrongSource (modellingRule, high) Iec61850DOAbbreviationLiteralsDuplicateDescription (modellingRule, high) Iec61850AssociationsThatShouldBePrivate (modellingRule, high) Available rules in AttributeValidator (category, severity): Iec61850DOAbbreviationLiteralsNeverUsedInDOName (modellingRule, high) Iec61850AssociationsWithDifferentEndVisibility (modellingRule, high) EnumLiteralsWithSuperfluousType (modellingRule, high) Iec61850DOAttributesWithSameNameDifferentType (modellingRule, high) AssociationEndsMissingDoc (documentationRule, medium) EnumLiteralsWithInitValue (modellingRule, high) Iec61850ConditionLiteralsNeverUsedAsConstraints (modellingRule, high) AssociationEndsWithBadDocStart (documentationRule, low) EnumLiteralsWithoutEnumStereotype (modellingRule, high) AssociationEndsWithBadDocEnd (documentationRule, low) AttributesWithInvalidMultiplicity (modellingRule, high) Available rules in OperationValidator (category, severity): AssociationEndsWithBadCharacterInName (namingRule, high) CimAttributesThatShouldBeOptional (modellingRule, high) OperationsWithUpperCaseName (namingRule, medium) CimAssociationEndsNameStartingWithLowerCase (namingRule, high) AttributesWithInvalidTypeNull (modellingRule, high) OperationsWithUnallowedStereotype (modellingRule, high) CimAssociationEndsNameShouldBePlural (namingRule, high) AttributesWithInvalidTypeString (modellingRule, high) OperationParametersWithUnallowedStereotype (modellingRule, high) CimAssociationEndsNameShouldBeSingular (namingRule, high) AttributesWithTypeIdMismatch (modellingRule, high) OperationsWithInvalidReturnTypeNull (modellingRule, high) CimAttributesThatShouldBePublic (modellingRule, high) OperationsWithInvalidArgTypeNull (modellingRule, high) Available rules in DependencyValidator (category, severity): AttributesThatAreStaticButNotConst (modellingRule, high) OperationsWithInvalidExcTypeNull (modellingRule, high) DependenciesWithUnallowedStereotype (modellingRule, high) CimAttributesThatAreNotStaticNonConstWithInitVal (modellingRule, high) OperationsWithUnallowedTagNames (modellingRule, high) DependenciesWithUnallowedDirection (modellingRule, high) AttributesThatAreConstNonStatic (modellingRule, high) OperationParametersWithUnallowedTagNames (modellingRule, high) DependenciesWithUnallowedTagNames (modellingRule, high) AttributesWithUnallowedStereotype (modellingRule, high) OperationsMissingDoc (documentationRule, medium) AttributesThatAreEnumsInNonEnumeratedClass (modellingRule, high) OperationParametersMissingDoc (documentationRule, medium) Available rules in DiagramValidator (category, severity): CimAttributesThatShouldBeReplacedWithAssociation (modellingRule, high) OperationsWithBadDocStart (documentationRule, low) DiagramsWithBadOrientation (formatting, low) AttributesWhoseTypeIsInformative (modellingRule, high) OperationParametersWithBadDocStart (documentationRule, low) DiagramsWithUnallowedStereotype (modellingRule, high) AttributesWithUnallowedTagNames (modellingRule, high) OperationsWithBadDocEnd (documentationRule, low) DiagramsMissingDoc (documentationRule, medium) Iec61850AttributesWithInexistingSibling (modellingRule, high) OperationParametersWithBadDocEnd (documentationRule, low) DiagramsWithBadDocStart (documentationRule, low) CimAttributesWithFlagInName (namingRule, medium) OperationsWithBadCharacterInName (namingRule, high) DiagramsWithBadDocEnd (documentationRule, low) AttributesMissingDoc (documentationRule, medium) OperationParametersWithBadCharacterInName (namingRule, high) DiagramsWithBadCharacterInName (namingRule, high) AttributesWithBadDocStart (documentationRule, low) IEC61850 only Copyright Tatjana (Tanja) Kostic July 2016
33 Validation: Console loggingExtract from validation log with base-small.eap UML model ERROR Found 3 CIM compound classes with
34 Validation: Fine tuningIf you leave scope property empty, the full content of the UML will be validated To validate only some top level packages (per IEC WG), specify them in a comma-separated list Example is for validating IEC61970 and IEC61968, everything else does not get validated Recommendation: Before releasing std model, do full validation, to ensure nothing has been broken Sub-options for validation, to skip validation for one or more type of UML element By default, nothing is skipped To skip validating something, set it to true: Example for validating only associations Sub-option for validation, to skip individual rules ... but skip rules reporting associations having doc, and association ends missing doc model.filename = base-small.eap validation.on = true validation.scope = WG13, WG14 validation.packages.off = true validation.classes.off = true validation.associations.off = validation.attributes.off = true validation.operations.off = true validation.dependencies.off = true validation.diagrams.off = true validation.rules.off = \ AssociationsWithDoc \ AssociationEndsMissingDoc Copyright Tatjana (Tanja) Kostic July 2016
35 Features: UML model statisticsNumbers and more. Copyright Tatjana (Tanja) Kostic July 2016
36 Statistics: Minimum configurationYou must specify the UML model file name and enable statistics. Copy your own model file(s) into the project’s input directory. model.filename = base-small.eap statistics.on = true Copyright Tatjana (Tanja) Kostic July 2016
37 Copyright 2009-2016 Tatjana (Tanja) KosticStatistics: Overview Currently, these kinds of statistics get logged to the console: Counts of UML constructs: classes, packages, diagrams, tags, etc. On CIM-specific constructs for classes and attributes (e.g., CIM data types, compounds, association ends, …) On IEC specific constructs for classes and attributes (e.g., LNs, CDCs, packed lists, operations, …), underlying modelling (e.g., classes and attributes with constraints, etc.), and since 10v10: DO name decomposition and inverse (usage of abbreviations by DOs) Tag names and where they are used Items with constraints Identified UML version classes and name spaces On actual (direct and derived) dependencies among packages Copyright Tatjana (Tanja) Kostic July 2016
38 Statistics: On CIM-specific constructsExample of CIM-specific constructs from base-small.eap: Total number of elements in the whole model Total number of elements per top-level package (per WG) Counts also informative elements Copyright Tatjana (Tanja) Kostic July 2016
39 Statistics: On IEC 61850-specific constructsExample of IEC specific constructs from base-small.eap: Total number of elements in the whole model Total number of elements per top-level package (per WG) Copyright Tatjana (Tanja) Kostic July 2016
40 Statistics: On actual dependenciesOn actual (direct and derived) dependencies among top-level packages (i.e., WG owners), through: Inheritance (e.g., from PowerSystemResource) Type of attribute (e.g., usage of datatypes, enums, primitives, compounds) Associations Hand-drawn package and class dependencies Class’ operation parameters and exceptions (UML of IEC61850 only) (CIM only) Inheritance from IdentifiedObject and dependency on stereotyped types from Domain package not shown on purpose: To show them as well, set the two ignore* properties to false, or leave them empty Copyright Tatjana (Tanja) Kostic statistics.cim.ignoreIdObjectInheritance = true statistics.cim.ignoreDomainClassAttributes = true July 2016
41 Copyright 2009-2016 Tatjana (Tanja) KosticStatistics: TODOs Configuration options to disable some of the output: sometimes it’s overwhelming, but may be also very useful Better presentation and potentially storage: In-memory representation of the UML from .eap file contains a lot of analysed information about the model It all gets logged with DEBUG level (i.e., stored in the log file at each run) – see examples below for ConnectivityNode, ApparentPower, BasicIntervalSchedule :05:21,984 [main] DEBUG UmlModel - (796) WG13 CIM class Topology::ConnectivityNode, 1 superclasses=[IdentifiedObject], 4 associations; associated classes (bi-directional): asTarget=[Topology::BusNameMarker, Core::Terminal] asSource=[Core::ConnectivityNodeContainer, Topology::TopologicalNode] … :05:21,984 [main] DEBUG UmlModel - (616) WG13 CIM Datatype <
42 Features: MS Word doc generation from UML (and from CIM XSD profiles*)AKA : A very, very big pain… Copyright Tatjana (Tanja) Kostic *being implemented July 2016
43 MS Word doc generation from UML: Minimum configuration*You must specify the UML model file name and enable doc generation Ensure profiles.docgen.on is not set to true (= leave it empty or set it to false) (until we have new implementation) Ensure docgen.word.useDocFormat = true You must specify also the input (template) and the output (result) MS Word file names. Copy your own model file(s) and template(s) into the project’s input directory. model.filename = base-small.eap model.builder = sqlxml docgen.on = true profiles.docgen.on = docgen.word.inTemplate = base-small-template.doc docgen.word.outDocument = base-small.doc docgen.word.saveReopenEvery = 12 docgen.word.useDocFormat = true docgen.word.analysePlaceholders = docgen.word.useHyperlinks = See slide "File and package placeholders" See slide "MS Word speed considerations" ** See slides "Hyperlinks" and "MS Word speed considerations" *** Copyright Tatjana (Tanja) Kostic * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class ** Started (but not finished) implementing faster MS Word docgen, that will be default *** Since v02v01 July 2016
44 MS Word doc generation from UML: OverviewTemplate file is a regular MS Word document (not Word .dot template): You put in that template jCleanCim-recognised placeholders (see next slide) They control what to pick from the UML model and print into MS Word document When generating documentation, jCleanCim will: Copy your template file into the projects output directory, created automatically the first time you run the document generation, Rename the copied file as given in the properties file, and Fill it with the contents from the EA model in place of placeholders found. You can safely run document generation several times with the same name of the output file, without overwriting existing output files: If the output file exists, jCleanCim will rename it by appending a unique identifier The disadvantage is that you will need to delete those discarded files from the output directory from time to time, but at least nothing gets lost without your control Resulting file is available in the project’s output directory. Copyright Tatjana (Tanja) Kostic July 2016
45 MS Word doc generation from UML: PlaceholdersJuly 2016 The tokens enclosed in curly braces are the names of UML elements designating what needs to be inserted in place of the whole placeholder. Ensure there are no spaces around the dots ‘.’ (that MS Word “smartly” introduces for you on copy/paste) Placeholders assume that the names (of packages, classes and diagrams-within-package) are unique within the model; otherwise, the first item is picked and that may not be what you want… startUmlDiagram.{packageName}.{diagramName}.endUml startUmlDiagNote{packageName}.{diagramName}.endUml startUmlAttribute.{className}.{attributeName}.endUml startUmlFile..endUml startUmlPackage.{packageName}.endUml startUmlClass.{packageName.className}.endUml startUmlstartUmlIec61850NsName.{className}.endUml (IEC *, for name space name) startUmlPresenceConditions.{packageName}.endUml (IEC , 7-3, 7-4xx, for presence conditions) startUmlFCs.{packageName}.endUml (IEC , 7-3, for FC table) startUmlTrgOps.{packageName}.endUml (IEC , for TrgOp table) startUmlSclEnums.{packageName}.endUml (IEC , 7-3, 7-4xx, for SCL enums) startUmlAbbreviations.{packageName}.endUml (IEC xx, for DO abbreviations) startUmlDataIndex.{packageName}.endUml (IEC xx, for data semantics tables) startUmlLNMapPackage.{packageName}.endUml (IEC , for Domain61850 tables) * * * Copyright Tatjana (Tanja) Kostic * since 02v01 July 2016 Copyright Tatjana (Tanja) Kostic
46 MS Word doc generation from UML: Attribute and diagram placeholdersPlaceholder from template gets replaced with some content If placeholder is not recognised or has typos, error gets printed instead of content template result Copyright Tatjana (Tanja) Kostic July 2016
47 MS Word doc generation from UML: File and package placeholdersIf problem with the placeholder for package (in the heading paragraph), only error gets printed instead of the package content Doc generation (in particular for UML package content, recursively) takes long: In template debugging phase, to ensure you have correct placeholders for packages, set property docgen.word.analysePlaceholders =true; This will skip (time consuming) printing of the full content of packages After debugging the template, leave docgen.word.analysePlaceholders empty to get the full content printed! template result Copyright Tatjana (Tanja) Kostic July 2016
48 Copyright 2009-2016 Tatjana (Tanja) KosticMS Word doc generation from UML: Diag. note and explicit class placeholders template Available since 02v01. Diagram description normally gets printed automatically after a diagram, from within the containing package or class. DiagNote placeholder inserts the description of the diagram : Useful for some extension models, if you want e.g. to have a two-column table with a diagram in one column and its description in another. Classes normally get printed automatically from within a package. Sometimes you may want to selectively print one or more classes. Class placeholder inserts explicitly the class content: Ensure to put this placeholder in a heading to get proper indentation. result Copyright Tatjana (Tanja) Kostic July 2016
49 MS Word doc generation from UML: IEC 61850 name space placeholderAvailable since 02v01. IEC has specific format how to create the so-called name space name from attributes in the name-space UML class. IEC61850NsName placeholder inserts that normative string properly formatted. (Note: work is in progress for CIM canonical model and profile namespaces) template result Copyright Tatjana (Tanja) Kostic July 2016
50 MS Word doc generation from UML: Data index placeholderRequired for IEC61850 documents Can be handy for CIM series as well, and for custom extensions Can be used for “model debugging” and consistency check Prints all usages of a name for attribute From base-small.eap example: attribute basePower used in classes BaseVoltage and BasePower Currently, only attributes If needed, (named) association ends could be added For this to work, you must specify the desired package name (or multiple comma-separated package names) in the property validation.packagesWithDataIndex For this base-small.eap example, we have set validation.packagesWithDataIndex = Core To print this index for base CIM, set validation.packagesWithDataIndex = IEC61970 template result Copyright Tatjana (Tanja) Kostic July 2016
51 MS Word doc generation from UML: Automatically added items (1/2)Sub-clause “General” To avoid hanging paragraphs (i.e., text without containing clause) Figure reference and text, and figure caption Automatically numbered, consistent with existing figures in the template Legacy look, if desired, needs to be enabled: docgen.word.introToFigureBefore = true New look (default) : Legacy look: Copyright Tatjana (Tanja) Kostic July 2016
52 MS Word doc generation from UML: Automatically added items (2/2)Table reference and text for attributes and association ends Automatically numbered, consistent with existing figures in the template Copyright Tatjana (Tanja) Kostic July 2016
53 MS Word doc generation from UML: Fancy IEC 61850 table formatsSome IEC tables want special formatting With table title Sub-headings With special reference and element counts for array types With special presence conditions (instead of UML multiplicities) Etc. Copyright Tatjana (Tanja) Kostic July 2016
54 MS Word doc generation from UML: HyperlinksAvailable since 02v01. From IEC community push: To enable this feature, set docgen.word. useHyperlinks = true Default is false, because it is very time consuming (due to a second pass, to replace bookmarks with hyperlinks)! Hyperlinks are generated to refer to any class (and some special enumeration literals) that are defined within one MS Word document E.g. CtlModeKind, BasePrimitiveCDC, MFsbo For classes not printed in the given MS Word document, there are no hyperlinks E.g. INT32U, VisString255 Copyright Tatjana (Tanja) Kostic July 2016
55 MS Word doc generation from UML: MS Word styles considerationsIEC templates contain IEC-specific styles. For custom extensions, you need not use these: jCleanCim tries to use IEC styles, and if those are not defined, it uses default MS Word styles – see next slide Essential: Use correct styles for paragraphs with figure and table captions in the template jCleanCim must deduce the number of figures and tables already existing in the template to calculate on the fly the correct numbering for new figures and tables (when inserting/appending the documentation for the UML model elements and diagrams) If jCleanCim throws an exception during document generation, it is very likely that the MS Word threw exception due to wrong / inexistent / negative number for the figure or table caption Note: We cannot check those numbers from within the code, because the MS Word automation API does not provide reliable access to them. In the worst case, when we catch an exception from MS Word, we attempt to gracefully exit, after closing both the MS Word document and the EA model file. Copyright Tatjana (Tanja) Kostic July 2016
56 Copyright 2009-2016 Tatjana (Tanja) KosticMS Word doc generation from UML: IEC styles mappings to MS Word defaults Extract from the code: IEC styles MS Word defaults para( "PARAGRAPH", "Normal"), fig( "Picture", "Normal"), figcapt("FIGURE-title", "Caption"), tabcapt("TABLE-title", "Caption"), tabhead("TABLE-col-heading", "Normal"), tabcell("TABLE-cell", "Normal"), h1( "Heading 1", "Heading 1"), h2( "Heading 2", "Heading 2"), h3( "Heading 3", "Heading 3"), h4( "Heading 4", "Heading 4"), h5( "Heading 5", "Heading 5"), h6( "Heading 6", "Heading 6"), h7( "Heading 7", "Heading 7"), h8( "Heading 8", "Heading 8"), h9( "Heading 9", "Heading 9"); Copyright Tatjana (Tanja) Kostic July 2016
57 MS Word doc generation from UML: MS Word speed considerations (1/2)MS Word documentation takes very long for big models ! Profiling of jCleanCim-01v04 and then jCleanCim-01v05 has shown that it is not due to Java code or Java-COM bridge It is certain MS Word operations that are slow In particular, inserting captions for figures and tables, takes exponential time as the number of figures and tables grows It is impossible to disable MS Word doing that numbering, while preserving ability to print tables of figures/tables Since jCleanCim-01v05, we significantly improved the speed From time to time, we close the output document, and reopen it This seems to reset the MS Word’s “numbering memory” Since jCleanCim-01v08, some more improvement: Only if your template has been saved as Office 2007/2010/2013 document (.docx), without compatibility options Programmatic disabling of field update seems to work only then Copyright Tatjana (Tanja) Kostic July 2016
58 MS Word doc generation from UML: MS Word speed considerations (2/2)Use the docgen.word.saveReopenEvery configuration option Supplied config.properties file contains the “magic” numbers for tested documents, as for our development environment Try to slightly increase/decrease the value to see whether there is any improvement in your environment and for your document size Enable hyperlink creation only for the very end of your development cycle (i.e., just before generating the final document), as it is very time consuming If possible, save your template as Office 2007/2010/2013 document (.docx), without compatibility options Disable change tracking in your template Copyright Tatjana (Tanja) Kostic July 2016
59 MS Word doc generation from UML: MiscFor IEC61850 document generation, more properties need to be set: See config/config61850.properties file and javadoc of the Configuration class jCleanCim doc generation design: The UML packages content to be printed in MS Word actually gets calculated and stored in in-memory objects before interacting with MS Word, on purpose: This processing is very fast (sub-second) It is also detached from the .eap file We now collect also content for creating XML documents Copyright Tatjana (Tanja) Kostic July 2016
60 Features: XML doc generation from UMLThis feature was driven by IEC61850 Web Publishing efforts (and works for CIM as well) Copyright Tatjana (Tanja) Kostic July 2016
61 XML doc generation from UML: Minimum configuration*You must specify the UML model file name and enable doc generation: Ensure profiles.docgen.on is not set to true (make it false or leave empty) You must specify also the two output (result) XML file names. Copy your own model file(s) into the project’s input directory: IECDomain.xsd schema is already available in the project’s input directory If you want to limit the scope (generate only some name spaces), use the scope variable (similar as for validation). model.filename = base-small.eap model.builder = sqlxml docgen.on = true profiles.docgen.on = false docgen.xml.scope = WG10, WG13 docgen.xml.outSpec = base-small-tool01v08-spec.xml docgen.xml.outDoc = base-small-tool01v08-doc.xml Empty value generates everything found in the UML model. This example generates only WG10 and WG13 name spaces Copyright Tatjana (Tanja) Kostic * UML of IEC61850 needs more than this, see config61850.properties and doc in Configuration class July 2016
62 XML doc generation from UML: OverviewWhen generating documentation, jCleanCim will: Copy the schema file to the output directory, created automatically the first time you run the document generation, Create .xml files as given in the properties file, Fill them with the contents from the EA model, and Save all the diagrams in the output/pics directory. You can safely run document generation several times with the same name of the output files, without overwriting existing output files: If the output files (or the schema file) exist in the output directory, jCleanCim will rename them by appending a unique identifier The disadvantage is that you will need to delete those discarded files from the output directory from time to time, but at least nothing gets lost without your control Resulting files are available in the project’s output directory. For a release, zip the two produced instance files, the schema and the pics directory. Copyright Tatjana (Tanja) Kostic July 2016
63 XML doc generation from UML: Specification and documentation XMLBy design, we generate two separate instance files: Documentation, with translatable (potentially formatted) strings with an identifier Specification, with normative, non-translatable content, with references to documentation identifiers One can have any number of translated documentation files (with correct identifiers) that can be combined with the specification content Copyright Tatjana (Tanja) Kostic July 2016
64 XML doc generation from UML: IECDomain.xsdBy design, we have a single schema with two elements defined: One for specification XML file, and one for documentation XML file By design, the schema supports mix of CIM and IEC name spaces Each family under a separate element (for both specification and documentation files) You can select what you want to generate Copyright Tatjana (Tanja) Kostic July 2016
65 XML doc generation from UML: StatusThis schema has been driven by IEC efforts, with objectives: to feed the IEC web-based publication, with the target, among many, to offer a way to download an XML machine readable file, representing the IEC data model definitions. to feed automatically a tool which would be able to semi-automatically check the compliance of some SCL configuration files based on the IEC data model. jCleanCim tries to support both CIM and IEC model management needs, so: Whenever applicable, a feature required by one family of standards gets implemented for the other family of standards If we one day finally come to some kind of harmonisation at the UML level, we are ready Feedback on the schema design for the CIM community is welcome and asked for: If there are experts in XSLT who would volunteer to provide stylesheets, that would be just great ! Copyright Tatjana (Tanja) Kostic July 2016
66 Features: CIM profiles vs. UML model cross-check*Copyright Tatjana (Tanja) Kostic *being implemented July 2016
67 Copyright 2009-2016 Tatjana (Tanja) KosticPerformance Copyright Tatjana (Tanja) Kostic July 2016
68 Copyright 2009-2016 Tatjana (Tanja) KosticTime considerations for reading from EA file and MS Word document generation Each feature that has been run ends with time elapsed logged to the console: EA and MS Word automation API implementations are terribly slow: Java processing for validation, stats, doc collection and XML doc generation is of order of milliseconds to seconds The “party breakers’” times are given on the next slide Times measured on ThinkPad T410 with Windows 7 64-bit, Java 7 32-bit, Office 2010, EA 9.3: So, plan your coffee and lunch breaks Copyright Tatjana (Tanja) Kostic July 2016
69 EA-dependent operations: Execution times with jCleanCim-01v08Improvements through new implementation enabled with option model.builder = sqlxml | db Full CIM model (iec61970cim16v17_iec61968cim12v06_iec62325cim02v07.eap): ~1520 classes, ~7680 attributes, ~1050 associations, ~90 dependencies, ~570 diagrams. Full IEC61850 model (wg10uml02v12-wg18uml02v10c-wg17uml02v09a-jwg25uml02v02a.eap): ~1670 classes, ~6250 attributes, ~85 associations, ~260 operations, ~380 dependencies, ~230 diagrams; plus tonnes of class and attribute constraints, tagged values, and markup in the documentation of elements. Small test model (base-small.eap): ~360 classes, ~700 attributes, ~95 associations, ~15 operations, ~70 dependencies, ~95 diagrams. EA-dependent operation 01v07 useSql=false / useSql=true / - 01v08 japi / sqlxml / db Open .eap file of any size (SSD hard disk) 5-10 sec / 5-10 sec / - 5-10 sec / 5-10 sec / sec Read CIM.eap (with docgen.on=true: ~355 exported diagrams) 3 min / 29 sec / - (+50 sec) 3 min / 20 sec / 7.5 sec (+50 sec) Read IEC61850.eap (with docgen.on=true: ~200 exported diagrams) 2.8 min / 26 sec / - (+32 sec) 2.8 min / 17 sec / 6.4 sec (+32 sec) Read base-small.eap (with docgen.on=true: ~93 exported diagrams) 34 sec / 10 sec / - (+12 sec) 34 sec / 8 sec / 1.3 sec (+12 sec) Copyright Tatjana (Tanja) Kostic July 2016
70 MS Word-dependent operations: Execution times with jCleanCim-01v08MS Word doc generation 01v07 (docgen.saveReopenEvery) duration 01v08 (docgen.saveReopenEvery) duration speed improvement IEC Base (637 tab, 67 fig) (12) 43.3 min (27) 9.5 min 4.6 x IEC (378 tab, 37 fig) (16) 12.1 min (24) 4 min 3.2 x IEC (637 tab, 40 fig) 36.5 min 7.9 min IEC Dynamics (387 tab, 178 fig) 18.5 min 6 min 3.1 x IEC including a subset of IEC , with special table formatting (134 tab, 31 fig) 4.4* min (5) 3.9 min 1.1 x* IEC , with special table formatting (244 huge tab, 40 fig) 41.4 min 27.5 min 1.5 x base-small (102 tab, 27 fig) 0.8 min 0.7 min 1.1 x** Copyright Tatjana (Tanja) Kostic July 2016
71 Copyright 2009-2016 Tatjana (Tanja) KosticInstead of conclusion Copyright Tatjana (Tanja) Kostic July 2016
72 Copyright 2009-2016 Tatjana (Tanja) KosticKnown issues Related to MS Word doc generation only – due to troubles with its automation API capability With some IEC templates, automatic update of tables of contents/tables/figures does not work properly Workaround: After doc generation, do manual update of all TOCs Incorrect numbering of captions in the auto-generated document Workaround: Before doc generation, first stop tracking changes, then do update of all TOCs. If you have deleted figure/table captions, ensure you accept those changes (because automation API returns them even when marked as deleted, as long as they are present in the template)! Word pop-up window "memory insufficient. Do you want to continue?“ Workaround: Before doc generation, in the input template, disable spell checking and change tracking Exceptions when using localised versions of MS Word, due to style names Workaround: Install English language pack in MS Office Planned to support non-English versions of MS Office in the next release If acceptable by IEC, we may want to go simply with some XML one day... Copyright Tatjana (Tanja) Kostic July 2016
73 Copyright 2009-2016 Tatjana (Tanja) KosticSuccess stories Generating official MS Word documents for CIM : IEC since Ed.4 (base CIM14) IEC since Ed.1 (DCIM10) IEC since CDV (market CIM01), and all EU profile documents IEC NWIP(Dynamics) Documentation of CIM extensions for various projects Also: IEC , IEC and IEC since their Ed.2.1 Work ongoing for IEC (hydro), IEC (DER), IEC (wind) and IEC (condition monitoring) And IEC (comm. network engineering), the first official document from the IEC61850 family with the data model automatically generated from UML Copyright Tatjana (Tanja) Kostic July 2016
74 Copyright 2009-2016 Tatjana (Tanja) KosticWant more features? jCleanCim is being developed on volunteering basis. Consequently: The policy is to first support the immediate needs of official IEC TC57 UML models’ editors (CIM and IEC61850 families), for their tasks of UML model management: For the IEC process, and, For the user community. This includes: Bug fixes New features that automate model managers’ tasks (e.g. doc generation for CIM profiles, CIM profiles-UML cross-check) New features that are easy to add (e.g., new validation rules, improved logging, better validation filtering and such) If you want more advanced features (like GUI): Get involved – any help is welcome ! Copyright Tatjana (Tanja) Kostic July 2016
75 Copyright 2009-2016 Tatjana (Tanja) KosticCredits Authors of all the cool open source libraries and tools we use (see readme.html). Lars-Ola Osterlund (ABB, Sweden) encouraged development of documentation generation for IEC UML model donated to IEC TC 57, to illustrate at least one benefit of using UML in WG10. Kendall Demaree (Alstom, US), while being CIM model manager in 2008, developed open source CIMinEA application that helped us move standard CIM from Rational Rose to Enterprise Architect, and which allowed document generation for IEC , Ed. 3. That work inspired some of the functionality related to document generation in jCleanCim. Hubert Kirrmann (ABB, Switzerland), Laurent Guise (Schneider Electric, France) and other members of the WG UML task force motivated development of the extended functionality for the needs of validation and document generation for IEC family of standards from UML being developed by the task force. Pat Brown (EPRI, US) and Christoph Fleischer (ABB, Switzerland) have been continuously providing a valuable feedback as users of jCleanCim, which resulted into improved quality, and enhanced and new features. Special thanks to Laurent Guise (Schneider Electric, France), who has been providing prototype implementation for a couple of new features and issue fixes – most of which have been ported to the jCleanCim architecture. Headway software, which provided open source license for their great tool Structure 101 (http://structure101.com/), used to keep the jCleanCim architecture clean. Copyright Tatjana (Tanja) Kostic July 2016