by John Ragan
|
Select A Section home public service database mgr. data access data modeler site notes |
Currently In This Section CoreModel Operation Manual ( Please Scroll Down ) |
Pages In Section summary installation architecture testing metadata svr. user manual |
|
Database Modeler & Metadata Server __________________________________________________
============================================
CoreModel Copyright 1996 - 2020 John E. Ragan.
__________________________________________________ Chapter Preface __________________________________________________
_________________________ CoreModel is unique in a category which is itself highly specialized and esoteric. Therefore, this document addresses CoreModel simultaneously at the mechanical and conceptual levels. It will be most easily understood by experienced professionals and computer science majors, but can provide excellent education for those who are willing to dig into it.
When CoreModel is installed, a copy of this documentation is installed. It can be viewed on screen within CoreModel by clicking on the doc button that is in every module. The documentation may remain displayed if desired. If you obtained a copy of CoreModel from a friend, and are reading this within CoreModel, you can find updated documentation and upgraded systems on the internet at http://CoreModel.com/ . There are export utilities in various places within CoreModel which allow the modeler or administrator to export the documentation to text files, web pages, and schema based XML (see below). If the CoreModel Web Administrator robot is activated in the Metadata Server, it publishes the documentation on its web site for access via internet or intranet. The CoreModel copyright covers this document and the document can be distributed only in its entirety with this statement and the copyright included as detailed in the license.
_________________________ CoreModel's copyrighted approach to modeling made theory its foundation. This insures that its mechanical structure will adapt to unforeseen real-world problems. The operational mechanics that the modeler employs within it are actualizations of the theoretical realm. (For the Master Modeler, CoreModel's theoretical background is discussed in the Advanced Topics section.) ( A graphical map can be seen on the internal architecture page.) The main module always loads first. There are five modules which are loaded from the main module. To alleviate confusion and to assist the modeler in following the workflow, each module must be closed before another can be opened. The four main work modules are conceptual, logical, physical, and implementation. It is in those four modules that the modeler does his work. The modeler moves his model to the next abstraction level as his work progresses. CoreModel is adamant about making sure that the modeler is always aware of the the level in which his model currently exists. However, to insure the ability to clean up discovered problems, each level allows the modeler to perform the functions of all the previous levels and he is even allowed to address his model at incorrect abstraction levels if he consciously decides to do so. Sitting to the side of the modeling modules is a fifth module which belongs to the system administrator. The administration module provides a means of supporting the many complex needs of the modeler so that he can devote his thought to creative modeling. Within the CoreModel system, there are a number of sub-systems which are each designed to perform specialized tasks. Architecturally, those sub-systems are not necessarily identified entirely within any module; some are entwined throughout the larger structure of CoreModel, and some even protrude from the main body to affect a world-wide presence for a CoreModel installation. It is not entirely necessary to be aware of sub-systems to operate CoreModel effectively and they can usually be thought of simply as utilities which are local to each module.
_________________________ There is varied opinion on the abstraction level at which various model domain objects should be introduced into the design process. Therefore, if you disagree with the placement of an object, please respond. For example, the table type might be introduced in the logical level, which is where it seems to me to be most appropriate and efficacious, but it could be introduced later.
_________________________ This is to define the installation noun and not the installation verb as it pertains to CoreModel. For purposes of this document and for communication between interested parties a CoreModel installation is defined by its database. Each CoreModel installation consists of one database regardless of the number of CoreModel workstations and CoreModel servers that use its database. Every CoreModel database constitutes a CoreModel installation. When CoreModel is installed on a computer, it becomes a CoreModel installation. If it is directed to a shared database, it ceases to be an installation and becomes a part of the existing installation, and is a CoreModel workstation or server within that installation. Geographical location and dispersion of an installation are irrelevant to the fact that it is an installation. The relation to a CoreModel installation of external systems is irrelevant to its identification as an installation. The kinds of machines and the architecture of the machines and their operating systems on which a CoreModel installation operates are irrelevant to its identification as an installation.
_________________________ CoreModel may be installed and used effectively by just one person. CoreModel is also designed to be used simultaneously by many people and many systems at the enterprise level. To facilitate discussion and to increase clarity, the documentation is written as though the CoreModel installation is being used by a team of modelers and is supported by a system administrator, but everything covered in the documentation is equally applicable to a single-user installation.
_________________________ The error handling mechanism is being remodeled to comply with the AxleBase protocol. At this time, the error number component of the return is not planned for implementation, but that decision may be revisited in the future. It is hoped that the AxleBase protocol will get a better handle on internal system errors.
__________________________________________________ Chapter Modeling Operations __________________________________________________
_________________________ In general, CoreModel reports on all major activities that the operator initiates. Reports are automatically sent to disk and to the screen in many cases. For some operations, the modeler may select one or the other or both. Report structure varies throughout the system. Reports may be of the modeler's current data state and they may include information on system operation. For example, a model validation report will contain information on the state of the model including specific data deficiencies which will be of interest to the modeler, but it may also contain reports of system errors that occurred during processing. An example of the latter may routinely be found in the model import operations which will frequently contain reports of errors in the import process. It is important that the modeler review all system output immediately after its creation. When a report may be needed for record keeping or for management reporting, it is saved in a disk file. The file is usually created in the default location unless the modeler specifies otherwise. The default location is the same for all modelers in the system and is selected by the CoreModel system administrator. The system usually reminds its operator of the location of the report file. A repetition of the same process will usually replace the old report with a new one, so if chronologically serialized progress reports are required, the modeler should be prepared to move or rename the files. The report file format is ASCI text so that it can be pulled into any editor for formatting or for inclusion in formal reports. The displayed report will usually be identical to the one that is in the file and will usually remind the modeler of the fact that the report may also be found in the disk file. For most operations, the report will be displayed at the end of the process, so that will be a signal that the process is complete. After viewing it, simply close the report screen. Although unlikely, it is possible for an operation to produce a report that is so large that CoreModel cannot display it. If that happens, he will simply notify the operator of the problem and remind the operator that the report has been saved in the file on disk. The report may then be loaded into a word processor. Most reports have the date and time of the report in the heading and the run time in the footer. The run time is accurate, but includes only the time required to run the process that created the report. It does not include such things as the time required to retrieve a large report for display.
_________________________ Before work on a model can begin, the system administrator must create a master model object which initializes the model. The model will be built within that object. The data modeler can then select that model and open it in the appropriate design module. There is no limit on the number of models that can be handled simultaneously by the system. Each model is managed independently, so it can be worked on at its own abstraction level, and work can be stopped at the appropriate level for each model. Each model master record will have a model level code assigned to it by the administrator when it is created. That code will be seen as a prefix to the model name in the main screen list. That code shows the current abstraction level of the model as it moves through the model abstraction levels.
_________________________ Unlike others, this system enforces very little. It is hoped that this freedom will be appreciated by the professionals, encourage the beginners to study by allowing them to get into trouble, and cause consternation among the suits. However, the system makes a whole-hearted attempt to provide a sound framework for development and delivery. Work may be stopped and delivered at any point. For example, a project goal may be to complete only a logical model or a physical model. But a project may be pushed through the implementation level if so desired. Be sure to look for the various tools such as the provision for naming standards enforcement. As you use those tools, they will prompt you as you step through them and will warn you when appropriate. If you have a model substantially designed and would like to experiment with it, but do not want to risk corruption of your beautiful model, have the administrator replicate the existing model to a new name. You can then try out new ideas on the copy. If the result is ugly, tell the administrator to delete it and make a new one. It is generally prudent to create model objects in the following order: tables, data domains, indices, columns. Of course, an iterative approach is the ideal manner as one thinks through the model. If you work with several screens open, be sure to refresh as you work. CoreModel is designed to allow the modeler to have numerous screens open within a module to simultaneously expose various parts of the model. The modeling process seems simple until one actually attempts it, and then one finds that all the numerous details and concepts can be overwhelming. For that reason, the modeler is allowed to display his model as he sees fit. ( If numerous screens are opened and a change is made to the data in one, the others should be told to refresh themselves. ) The data browser screen is designed to allow various parts of a model to be seen from customized perspectives. For example, in the logical model, one may scroll through all of the fields in the model or select all the fields in a certain table. When browsing the fields, one might select a field to edit in the edit screen. The modeler might simultaneously browse through the tables or other objects.
_________________________ First of all, the modeler should insure that the CoreModel system administrator is prepared for modeling to begin. The administrator must enter large amounts of various kinds of data into system tables that will be used by the modelers. If the data is not there when it is needed, the modeler's creative thought processes will be disrupted. When ready to begin, the modeler will ask the system administrator to create the master model object. That object is the framework within which all of the modeling activities are based. The modeler may need to sit with the administrator as he creates it to provide various pieces of information about the model. When the master model object is created, the model springs into existence. After the master model object is created, the modeler will load the CoreModel system on his workstation, and the name of his model will be seen in the list on the main screen. It can be loaded either by double clicking its name or by highlighting the name and then pressing the open button. CoreModel will check the model's abstraction level, load the model into the correct module, and open the module. ( If the modeler wishes, he may over-ride that behavior by selecting a different abstraction level from the list on the right side of the main screen, and then telling CoreModel to open the model. At this time, that may seem pointless, but as the modeler becomes proficient with the tool, this feature will become of benefit. ) At that point, the model is now open in the modeling module and work may begin. If the modeling was begun at the conceptual level, create the required entities, then create the attributes. Report, validate, modify, etc. When the modeler is confident of the state of the model at that level, CoreModel can be told to move it down to the next level, and then continue its development.
Modeling Operations CoreModel includes a built-in demonstation model which contains examples of all objects and facets of the system. It is automatically created when the system is installed. The Administrator has utilities to destroy and recreate it for the modeler, so it can be freely used for experimentation. The administrator can also rename it so that multiple modelers can have copies and so that it can be saved. The demonstration is a model of the 2002 CIA World Factbook as it was presented on the web site. It contains over a hundred tables, with hundreds of columns and associated objects as though a modeler had spent many hours building it. The demo is complete enough for immediate instantiation. However, some errors were intentionally left in it to test various modeling skill levels. Also, since target database servers vary considerably, minor changes may need to be made in it for some servers. For example, the data types may need to be changed. ( Those who work alone and provide their own system administration will find additional discussion of the demonstration model in its own section in the administration portion of this manual. )
_________________________ The system interface is designed to reflect the modeling process, so it contains a separate operational module for each model abstraction level. Each module guides the modeler in developing a model at a particular level. The modules are designed to assist and not to dictate, so the development of a model may begin at any level. A model always opens in one of the modules. It can be addressed only by viewing it within an abstraction level. It will normally open at its current level in the module for that level. However, that behavior is easily over-ridden. For example, if the model is at the physical level, the modeler can highlight the logical level on the main screen and the model will open at the logical level. The modules reflect the progression of the modeling process from extreme abstraction down toward the reality of actual data. If a model is first addressed at the extreme abstraction, it can be reduced from level to level by telling the system to process it to the next level. Work can then be resumed on the model at the next abstraction level in the new module. Each module is designed to follow the logical construct expansion as the model moves downward, so each one provides the tools specific to the current level. The modeling modules are:
Professional modeling begins at the conceptual level. Beginning at that level and then stepping through each of the levels and completing the tasks at each level eases the modeler's work, avoids overlooking tasks, and assists in attaining a quality professional product. However, CoreModel allows commencement and delivery from any level. An abstraction change is irreversible, but the modeler need not be concerned about becoming locked into a decision by moving to a new level. CoreModel is flexible. If something was overlooked in the previous module, it can be added or repaired in the current module. Also, CoreModel allows opening a model in an inappropriate module. When work is complete at an abstraction level, the step button will process the entire model object, with all of the objects that it contains, into the next level. It is important to understand that CoreModel does not lock the modeler into a level. If work has progressed to the day of implementation, and the model suddenly needs another object, it can easily be added. Or if the modeler is working at the physical level and suddenly feels overwhelmed by the complexity, he can easily open the model for inspection in the conceptual level. Furthermore, CoreModel's design prevents the corruption of a model by working at an incorrect level. CoreModel is designed specifically for data modeling and manifests only the four levels of abstraction that are generally recognized in the industry. However, for those who require a higher level of abstraction in the modeling process, CoreModel can be forced to manifest a genesis level module. This is not recommended and is not encouraged, but the ability is provided because it is sometimes needed in the project planning phase. The genesis level is not listed on the main menu, by design. The only way to work on a model at the genesis level is for the system administrator to assign a new model to that level when the model object is created. The only object that is addressed in the genesis level is the protoform. The genesis level is best understood by studying the functions and use of the protoform object. Moving from the genesis level to the conceptual level is irreversible because the movement irrevocably consumes the protoform objects of that level to create the objects for the conceptual level. However, if the modeler finds after moving to the next levels that he needs to address issues of the genesis level, the tools are, of course, in place at those levels.
_________________________ Because CoreModel is a full featured system, there is a lot of information about the data screens in this section. Various tools available in the screens will be pointed out, but a beginner should not become bogged down in the details. There are three types of data screens in CoreModel. They are identified by function as the create, edit, and browse screens. Modeling is a creative process which, like all creative processes, requires some freedom. CoreModel is designed to allow the modeler to fiddle, doodle, and think, so the modeler may simultaneously open as many screens in a model as he wants. The only restriction is between the create and edit screens. To avoid confusion, CoreModel avoids allowing a create and an edit screen to be open for the same object. All of the screens may be loaded by first highlighting an object and then pressing one of the labeled buttons. For example, in the logical level module, highlighting the column selection, and then pressing the create button will open the create screen for the column objects. Multiple objects may be loaded simultaneously. For example, if the modeler begins work today on a model to which he is adding finishing touches, he might highlight all model objects and press the edit button to open edit screens for all of the objects. Finally, in addition to the other methods of opening the browse screen, double clicking on a highlighted object will open the browse screen for that object. An info button on every data screen provides context-aware information. The information screen is independent of most operations, so after it is displayed, it will usually stay up until closed.
Modeling Operations The create screen is used to create model objects or data records. When it is displayed, it is altered to fit the selected object. For example, a column create screen will have data types and a table create screen will have table types. The appearance of a create screen may also be dependent upon the model abstraction level. For example, CoreModel will put a tuple count on the table screen at the implementation level which will be absent at the logical level. Using the create screen is a matter of tabbing to each field, filling in the blanks, and pressing the save button. When the save button is pressed, if the save operation goes well, the screen will be cleared to allow another object or record to be saved. If there is an error in the data or if an error occurs in the save operation, the data will remain on the screen and CoreModel will display an error message.
Modeling Operations If the edit screen is opened for an object type that does not yet have any objects created, CoreModel will complain that there is nothing to edit and then close the screen. When an edit screen is opened for an object type, CoreModel makes all of that type of object available as a set beginning with the first object, and the data for the first object is displayed on the screen. The numbers at the bottom of the edit screen tell how many objects there are in the set and which object is currently displayed. The arrow keys at the bottom of the screen move through the set. For example, clicking on the right arrow will move to the next object and display it on the screen, and holding that arrow down will rapidly advance through the entire set. The cursor may be placed in a field to edit whatever is in that field. Saves are not automatic. Changes must be saved by pressing the save button at the bottom of the screen. Changes can be undone with the undo/clear button if they have not been saved. The delete button effects only the currently displayed object or record. When it is pressed, the current object is deleted from the set. ( After deleting an object, if other screens are open that use or reference the destroyed object, be sure to refresh them to remove the object references from them. ) The edit screen can search for a specific object. Place the cursor in the field that will be searched and press the find button. CoreModel will prompt for a value for which to search in that field. If the value is found in an object after the current one, the focus will move to that object. Any string of characters such as a partial word or number or a group of words may be entered because the search is conducted for a string of characters. The search is designed for ease of use, so the string is not case sensitive.
Modeling Operations Whereas the two other data screens are primarily oriented toward working on the data, the primary purpose of the data browser is to display information and to display it in an easily red format through which the operator can quickly scan. It is designed to display entire object or data sets and to compact a lot of information onto the screen. For example, the data browser will display all of the tables in a model with one table per row and all of their characteristics organized in columns. The data browser is primarily designed to simply allow the placement of vast amounts of information on the computer screen, and to do it quickly and simply, so a knowledge of its many features is not required to benefit from its greatest asset. However, after learning how to use those features, the modeler can display, sort, select, etc. all of his data as he needs it. All data browsers look the same except that CoreModel puts different titles and column labels on each. CoreModel watches internal events and conditions. When the operator requests a data browser, CoreModel checks the identity of the operator and the characteristics of the proposed data set. Based upon matched criteria, CoreModel decides which function keys can appropriately be activated on the data browser as it is displayed. For example, the browser's delete key will be deactivated when displaying data that is maintained by the system administrator. - Record Numbers in the Data Browser - Cell Size in the Data Browser - On-Screen Edits in the Data Browser - Edit Screen Loader from the Data Browser - Delete in the Data Browser - Refresh of the Data Browser - Find in the Data Browser To find a blank field, enter two adjacent apostrophes. A negative search may be made by entering *not* followed by the string such as *not*suzanne-. Entering the string *not*'' would find objects where the target field is not blank. To find names that sound like something, enter *sounds* followed by the string of characters. Entering *sounds*cindi will find cindi, cindy, Cindi, and sIndy. This search is very slow on large data sets. At best, it works poorly for English and, of course, is useless for Spanish and other foreign languages. - Selects in the Data Browser The select is not case sensitive unless the database is case sensitive. Behavior of the select operation is determined by the type of data in the selected column. Numbers and dates must match exactly, but a character string can be all or part of the target string. - Quick Sorts in the Data Browser - Extended Sorts in the Data Browser - Fonts in the Data Browser - Close Button in the Data Browser
Modeling Operations The use of numerous drop down list boxes in certain fields helps maintain the integrity of a model. If a list is blank or does not contain a needed item, then the modeler or the administrator needs to add the needed item to a lookup table, and then reload the form, module, or system so the item will be displayed in the list box for selection. For example, if the modeler is creating a column, and finds that there are no data types from which he can select, then the system administrator needs to build a table of data types. After the data types are entered into CoreModel, the modeler can reload the system or press the refresh button on the column screen to load the new data types. Some of the lookup data in the modeler's screens is actually entered by the modeler. For example, the column edit screen has a drop down list of tables from which to choose the table in which the column belongs. That list of tables, of course, comes from the tables created by the modeler. A powerful ad hoc lookup list can be created simply by displaying a data set in the data browser. As the modeler works, he can open as many data browsers as he needs for reference. Data can be sorted and selected as needed, and the data browser will search for information.
_________________________ In CoreModel, the model is a master object which is also composed of objects. Some of those objects are, themselves, composed of objects. This section addresses those objects, what they are, how they are created, and how they are manipulated to create the characteristics of the model object. The system administrator first creates a model object, and then hands control of it to the modeler who develops the model by creating its internal objects and by fashioning all of the object characteristics through their respective attributes. Each object has various attributes that are peculiar to the object and which determine the object's characteristics. The process of building a model should be left to the discretion of the modeler and CoreModel complies with this by trying to be as unobtrusive as possible. CoreModel assumes that the modeler is a professional who may create and edit objects in any order that he wishes. The modeler is free to move between objects and levels as freely as he wishes, and to create and edit objects as the creative fancy moves him. Although CoreModel provides assistance in many ways, he attempts to create an environment of freedom for work and creativity. This table shows where each object automatically appears.
Modeling Operations The object manager objects are not part of the model. They are memo objects which are provided by CoreModel as assistance to the modeler. They may be entirely ignored by the modeler. The manager objects could be especially useful in large projects where numerous people impinge upon the work. The object manager object establishes ownership of an object. There is a one to one relationship between an object manager and an object. ( If the objects were managers, there would be a many to one relationship. However, CoreModel manifests the object as specific to a model object so that it can carry the object's attributes as described below. This makes the relationship one to one. ) The manager provides a means of setting extended characteristics of an object and of recording additional notes about the ownership and management of the object. If the modeler wishes to create an object manager, CoreModel will carry the manager through all levels as an attachment to the object.
Modeling Operations Let us assume that work on the model has begun at the conceptual level as usual, where the entities are equated with business objects. It then becomes most efficient to quickly specify and name the entities, so it would be best to create the entity objects first in a broad pass at the model. In that case, highlight entities and click on the create button to open the entity create screen. Create an entity by entering a name and description and clicking the save button. After creating the entities, open the attribute create screen. The name field can be filled in, or it can be selected from the drop down box which lists the data domains that have been created in the system. It is possible that the CoreModel system administrator has established a number of data domains which are used throughout the organization, so the modeler can sometimes select some of them as attributes for his model. Some organizations require that data domains be observed. The entity name can be entered on the attribute screen, or it can be selected from the drop down list which shows all of the entities in the model. The entity edit screen has an attribute button. Pressing it will display all of the entity's attributes in the data browser. The entities and attributes will become the model's tables and columns. It is prudent at this point to insure that every entity and every attribute has a brief description. This is far easier to do now than later when there will be many other things to worry about. The descriptions can be critical in large organizations where the metadata server is used. When the work is complete at the conceptual level, the step button may be pressed to step the model down to the next abstraction level where work will be resumed. When CoreModel is reloaded, the model will be waiting in the logical abstraction level. All of the entities and attributes will have become tables and columns.
Modeling Operations At the logical abstraction level and higher, the modeler can create his own data domain objects. It might be helpful to immediately create all of those that the modeler knows that he will use before other object types are created. Highlight the data domain selection and press the create button to open the domain create screen. It would be beneficial at this point to insure that a description is entered for the domain before it is saved because other modelers may later need to use it and the modeler may use it in future projects. ( Data domains are the only objects and information which are allowed limited inter-model domain communication. ) The domain data type may be entered or selected from the drop-down list. The drop-down list contains all of the data types which were entered by the system administrator. If the required data type is not in that list, it might be beneficial to ask the administrator to enter it because it will probably be needed again. The data types are target specific. Domains can be created within each model to be used within their respective models. Additionally, standard domains may be created by the system administrator which may be used throughout the system in all models. The administrator can acquire ownership of an existing domain by displaying and saving its record, and it will then be available to all models.
Modeling Operations When the model moves from the conceptual level to the logical level, the entities are translated into table objects, and all that is needed is enhancement and expansion of those objects. If it is found that a new table is required, it can be created in the new level. CoreModel is designed to facilitate this kind of iterative process; it is designed to facilitate creativity and not to enforce perfection, because cleaning up a model with a good design is much easier than fixing a bad design. Normally, the table objects should be created before the columns. The table edit screen has a unique button for columns. Pressing it will display all of the table's columns in the data browser. As the model is processed through abstraction levels, the modeler will find that the table objects acquire additional attributes because not all data elements can exist at all abstraction levels. For example, a tuple count is meaningless above the physical level, so it will not be encountered before that level. Each time that the model is stepped to the next level, the modeler should review each object in the new context, and in that light, alter data that he entered previously and add new information.
Modeling Operations When the model moves to the logical level, the attributes are translated into column objects, and all that is needed is enhancement and expansion of them. If it is found that new columns are needed, the iterative process allows them to be created in the new level. Normally, the columns should be created after the tables are created. On the column add and edit screens, a column name may be typed into the name field or it can be selected from the drop-down list of data domains. If a data domain name is selected in the create screen, the other elements of the domain will be written into the column fields. Like the tables, as the model is processed through abstraction levels, the column objects acquire additional attributes because not all data elements can exist at all abstraction levels. Each time that the model is stepped to the next level, the modeler should review each object in the new context, and in that light, alter data that he entered previously and add any new data. The column width appears at the physical level. It is not required for model validity and is available for management and forecasting purposes. Experienced modelers have been surprised so often that they value such niceties. The nullable and unique characteristics also appear at the physical level. They are sometimes required. Otherwise, they may be ignored. At the implementation level, the column default value appears. It is not required, but may be used if needed. CoreModel cannot check the data type of a default. He enters apostrophes and parentheses in the field to remind the modeler to consider including them if needed by the data type. The edit mask is a memo only entry for communication with system developers who may use the model. It's appearance in the metadata server can be a big plus for any project. The ordinality attribute of a column is not common in the profession. CoreModel supports ordinalism for the benefit of the modeler to allow sequence control of column instancing. Ordinality is not required and may be left at zero for all columns if the modeler so desires. The source and source type appear at the implementation level. When they appear in the metadata server, they may provide the organization with deep information about the nature of the data, and can give management critical insight into system linkage.
Modeling Operations The relation object is an actualization of the relation concept and is not required in a valid model. It is provided to help the modeler study the model and to help him insure the validity of the model. Although relation objects are not required, their use is strongly recommended. They are required for the use of the validator utility. Relation objects cannot be created until the tables and columns are created. A name may be entered for the relation, or it may be left blank and CoreModel will name it. There are two identical panes on the left and right sides of the relation screen. Each pane corresponds to a side of the relationship. There is a table field in each pane which must be filled by selecting one of the table names in the drop down lists. The lists are filled with the names of the tables that the modeler has already created. Selecting a table causes its column names to be written into the column drop down list. If the table name is changed, the column name may need to be re-selected. The cardinality and type are not required, but their completion is recommended when the object is created. The description is provided for the modeler's notes. CoreModel can help with creating the relations. The relation utility (covered elsewhere in this documentation) in the modeling module will attempt to identify and manifest all latent model relations as relation objects. The operation is dependent upon the care with which the modeler constructs and names his objects. ( The relation utility is an example of the fact that in the professions of system development, the modeler must be aware that his data entry is a communication with systems that can understand things only in a literal sense, so he should take great care in naming objects. ) The obfuscation and warping of knowledge that goes on in the industry is rediculous. The monopolists have made it seem as though a relation is a hard coded thing, and have thereby confused many simple minded modelers about the true nature of the relational model. CoreModel attempts to offset that confusion by not providing a means of instantiating a relation. It simply will not do it.
Modeling Operations An index is an important object in the model and must be created and saved by the modeler as part of the model. It may be preferable to create all of the tables and columns first, and then the index objects. This will insure that the drop-down lists will be filled on the index screens. When the index screen is opened, all of the table names will be written into the table drop-down list. The column list will be blank. When a table is selected, the column list is emptied and populated with that table's columns. There is a column field and a columns (plural) field. The columns field is a list of the columns in the index and will be saved as part of the index object. The column field is for lookup only and will not be saved. It assists the modeler with filling the columns field. The columns list is a free form text field, so the column names may be typed into it, or if a name is selected in the column list, the system will write it into the columns list. The names must be comma delimited in the columns field. Formatting is permitted.
Modeling Operations View objects don't appear until the model has progressed all the way down to the physical abstraction level. Highlighting the view selection and pressing the add button will open the screen to create them. It is important to remember that CoreModel will not check object references and will not check syntax in views. This is not a CoreModel shortcoming, but is actually a feature to overcome the shortcomings of various database servers. Some deviate greatly from the industry standards; some cannot even accept an order-by clause in a view. Do not use the keywords select, as, from, where, or the view parentheses. A name is required. The view field will contain the resulting view. In it enter a comma delimited list of the column names that will be returned by the view. In the select field, enter the names of the columns that will be selected. If more than one table is represented, the columns must be qualified. Aliases are permitted. Do not enter the select keyword in the select field. In the from field, enter table names, linked tables, and link methods. Do not enter the from keyword. Selection criteria may be entered in the where field. Do not enter the where keyword. If CoreModel is asked to instantiate a model, he will supply the keywords when he parses and translates your work. The with option check and the description are not required. CoreModel will allow limited formatting within each clause field so that DML can be formatted, but formatting is not required. For extremely large blocks of text which cannot be comfortably worked on in the windows provided, CoreModel will participate in cut-and-paste operations. Quotes and apostrophes (sometimes referred to as single quotes) may be used as desired in a view definition. CoreModel will translate them into and out of its own database files. If an attempt is made to save a view record with nothing in a required field, CoreModel will ask the modeler to verify that that is the intention, but he will not enforce validity. This is by design to allow the modeler to work on views in a piecemeal fashion.
Modeling Operations CoreModel manifests the constraint concept in the form of constraint objects. For simple models, constraint objects may not be needed and may simply be ignored since simple constraints may be implemented within each object through its on-screen parameters. However, if complex constraints are needed in a model, the parameters provided for the other objects may not have the required sophistication. In that case, the more complex constraint objects can be created after the parent objects are created. In addition to the requirement discussed above, the modeler or organization may also prefer the readability and maintainability that are provided by CoreModel's constraint objects. In that case, all of a model's constraints might be affected through the constraint objects, and constraint parameters could be ignored on the parent object screens. If that strategy is followed, a model's behavioral characteristics might be more easily ascertained when reading published metadata, and maintainability might be greatly simplified, especially in a complex modeling environment. The operation of the constraint screens is forced to be fairly complex because of the complexity of the constraint concept. The edit screen appears simpler than the create screen, but that is slightly misleading because more of the complexities of the subject are manifested on the create screen. The apparent complexity actually arises from the inherent complexity of the constraint concept which is used for many tasks of great variety. The constraint screens actually simplify the constraint complexity. The modeler who occasionally creates a constraint may find that that convoluted task is simpler when done within CoreModel. In both the create and the edit screen, when the screen is opened, the names of all of the tables in the model are read into the table drop-down list, and the column list is still blank at that point. When a table is selected, the column list is emptied and re-filled with the names of the columns in that table. To relieve some of the complexity, when the constraint create screen is displayed, few of its fields are visible. CoreModel watches the modeler enter data and select options, and displays fields that need to be filled. CoreModel requires that each constraint must have a unique name because it is an object within the model. After entering the name on the create screen, the modeler must select the table or the column option to specify whether it is a table type constraint or a column type constraint. That selection will cause the display of the table and/or column field. If it is a table constraint, then the table field will be displayed. If it is a column constraint, then the table and column fields will both be displayed. Selection of the table or the column option will cause the display of the appropriate sub types of constraints that can be created for that selection. Selection of one of the constraint sub types will then cause the display of the fields that should or may be used in constructing the constraint. If the index option is selected, the index field will be displayed. When it displays the field, CoreModel copies the name of the constraint into the index field as a recommendation, but that may be changed. When the index field is displayed, the text field is also displayed and a comma delimited list of column names must be entered into it. Cut and paste may be used to get the column names. When the default sub type is specified, the default field is displayed in which the default value should be entered. For more complex constraints, the text option may be selected and only the text field will be displayed to allow the modeler to enter free form text. Extremely complex constraints may be worked on in a word processor and then copied into the text field Like other objects, notes are a recommended part of the constraint object. When a new constraint object is created, it's type and sub type are saved as object characteristics. When the constraint is later loaded into the edit screen, the type and sub type characteristics cannot be altered.
Modeling Operations Processes, of course, do not appear until the model has moved all the way down to the implementation level. There is little that is required in a process object other than the name. Let us note here that most processes found in the business world have little to do with the databases that they impact. Processes belong to external systems and were moved into databases as a means of management control. Therefore, processes belong more in the realm of programmers than in that of the data modeler, and the result is neither good programming nor good modeling. However, having noted that fact, the data modeler does live in a world where he must frequently build processes into models, so CoreModel supports them. A process object may be created in a model as a place-holder to show that the process will be implemented, but with the intent of not creating it initially. A place-holder process may be created simply by entering the name and description with a blank body. The fact that required data elements are not present will cause CoreModel to bypass it. Note especially that CoreModel will attempt an instantiation only of process types sp and tr. A process object does not need to be supported by the target. If it is not supported, instantiation will bypass it and note the action. That allows the process to be present in the model for publication and management review. For example, the model might be dependent upon an external pre-processing program written in COBOL (or whatever) and which the modeler wants to note within the model. Again, the fact that a process is not of type sp or tr will tell CoreModel to not attempt an instantiation. Do not enter the key words create, procedure, or trigger. Enter everything that falls after the process name. CoreModel will allow some text formatting within the body to support DML formatting, but it is not at all required. For extremely large blocks of text which cannot be comfortably worked on in the window provided, CoreModel will participate in cut-and-paste operations. There is a felt inconsistency in the system because view objects have several detailed entry fields and all of the key words are handled by the system, whereas the body of a process is entirely free form. The reason that a process format is so uncontrolled is that processes are so generalized, usually belonging in the world of programming, as previously noted, that CoreModel must intentionally allow the modeler to be uncontrolled in this area. Quotes and apostrophes (usually referred to incorrectly as single quotes) may be used as desired in a process definition. CoreModel will translate them into and out of its own database files.
Modeling Operations After a perfect model's instantiation is put into production, reality usually insists upon declaring that it is imperfect. Furthermore, the systems and people who use it tend to insist upon corrections. Since it is not prudent to make quick changes to an operational database, CoreModel provides the modeler with a means of notifying the organization of a pending change. At the implementation level, CoreModel introduces the change warning. The change warning has nothing to do with modeling, and should be ignored during the modeling process. It is functional after a model's instantiation is placed in production. It is an instrument of communication between the modeler and whomever may be using an instantiated model. The actual communication is affected by the metadata server. When a change warning is created, the metadata server sees it and begins publishing it on the web site and in the data dictionary. A change warning is obviously not a legitimate part of the repository, so when a subscribing system sees it appear in the dictionary, it serves as a flag for required action. The change warning requires no name. It is identified by the system when it is saved. The identifier cannot be changed. The name of the object that will be changed must be entered. A qualified name is recommended so that the subscribers can identify the target of intent, but not fully qualified since the warning is always identified with the model. For example, "column.table" would probably suffice. The target date is required and must be after the entry date. The target hour and target minute may be used, but are not required. The metadata server will continue to publish the change warning until target date plus two days. The description of the change that will be made is required. It must be less than two hundred characters to encourage brevity. The contact is actually a protest contact, and is required. This is the person, with contact information, who should be contacted to lodge a protest. It must be less than two hundred characters to encourage brevity.
Modeling Operations Within the business community, some interest has recently developed in the concept of the business object. Although the concept has not yet been definitively developed by the community, the business object is an attempt to create an entity that corresponds to an abstract business entity which represents a portion of the business enterprise. The business enterprise is a subset of mankind's various organized activities, all of which spring up to accomplish varied goals. The name "business object" is a misnomer because that name attempts to place the object within the subset of the business community. The object which is referred to by the "business object" name is actually an object of all abstracted activities and not just of businesses. Government enterprises, military enterprises, university research enterprises, etc. have the same activities which may be identified by the same objects, but to say that those enterprises use business objects would tend to confuse the entire issue, especially for the hundreds of thousands of professionals who never work in a business. Therefore, the concept of the business object is subsumed under the larger concept of the protoform object. The business object concept remains viable, but since it is only a subset of the many kinds of systems for which CoreModel is designed, and because its characteristics are all manifested in the protoform, only the protoform is presented in CoreModel. Please refer to the protoforms section for the use of business objects within CoreModel.
Modeling Operations Before proceeding with this subject, it should be noted that when protoforms are used, they are used at the beginning of a project, so this explanation is out of sequence since it is at the end of the object list. To follow the flow of the modeling operations, this section should be at the beginning of the list. However, that is why it is at the end; the protoform is not normally a part of the master modeler's concern. The protoform falls within the domain of management analysis and planning which would normally be done before handing the project to the modeler, so the protoform does not belong within the CoreModel sphere of interest. However, some interest has recently arisen in including the concept within the modeling realm, so CoreModel permits, but does not encourage its use. If the protoform is used, it provides a vehicle for passing the initial project concepts, thoughts, and findings to the modeler. When a need is addressed by management, and before it is developed well enough to hand to the modeler, management must wrestle with the identification of those things that may be included in a future model, and which will define the outlines of the modeling project. Those things may be facets, areas, or chunks of an organization's activities, data, or knowledge, or those things may, in fact, not actually exist, but may be conceptually created by management as a means of controlling real abstracted aspects of the organization or operation. Those are the things which are manifested in protoforms. Since this is a system documentation for the master modeler and is not intended to be a tutorial, the protoform concept will not be fully developed in this documentation. Those who have participated in the management initiative of projects know that the protoform may represent a massive flow of communication, huge files of documentation, long meetings, and many concepts and ideas that must be digested into concepts that are precise enough to hand to the technicians. The protoform construct is not intended to handle that monumental enterprise level task; it is the construct that is developed from that task. That construct is actualized in CoreModel as the protoform object, to which this document is addressed. The protoform object appears only in the genesis module. It contains generalized characteristics which are designed to initiate model objects. When the protoforms for a project are finished to the satisfaction of management, and the step button is pressed to step to the conceptual level, each data protoform is consumed by the process to cause the creation of an entity and an object manager. The name and object notes of each will go into the name and notes of an entity, and the name and the rest of the information will go into a manager. Since the protoform is consumed in the process, there is no possible reversal of this process. That behavior is by design to insure that the modeler is unencumbered by extraneous management pursuits. In CoreModel, all activities and all data flow downward into the final model, and in those models, there is no such thing as a protoform. ( The level changer process digests all of the protoforms and moves their information into modeling objects in the next level. Since the digestion of non-data protoforms produces nothing to pass on to modeling objects, all non-data protoforms disappear in the processing. The modeler should export protoform objects before processing in order to save the non-data objects. )
_________________________ CoreModel contains a number of utilities that are designed specifically to assist the modeler. All are accessed from within the modeling modules. Some, such as the name standardization utility, may be used from within any module. Others, such as the size forecast utility, are limited to those modules in which they are appropriate. In all cases, the utility is activated by pushing its labeled button
Modeling Operations There is a step button in each model module. When work is complete at an abstraction level, pressing the step button will process the model object into the next level. This process cannot be reversed, so the modeler should review all work before pressing that button. However, each level allows for correcting errors in the preceding level. When the model is moved to the next level, the system needs to insure that everything is finished and reinitialized. Therefore, it will shut down after the process is complete. When the modeler reloads the system, the model will be at the new level.
Modeling Operations For some organizations and for some professionals, standardized names are very important, but it can be difficult to avoid mistakes in the hundreds and sometimes many thousands of names in a model. Therefore, CoreModel provides a name standardization utility to assist the modeler. A name standards table is maintained by the system administrator in the administration module. The standardization utility compares all of those standards to all of the names in the model. Operation
The none option is a null selection and not a directive. The delimiters are exclusive. Zero is a null selection and not a directive. The prefixes may be any string of non numeric printable characters. The standardization utility may be run at any time in any abstraction level. The change of all of a model's names to comply with a naming standard is such a major change and could so badly corrupt a model that CoreModel is no longer allowed to actually make the changes. Instead, he reviews the model and gives the modeler a report of all discovered deviations. Since CoreModel does not understand English, he is unable to adequately handle hungarian notation. If hungarian notation is employed, CoreModel will attempt to be of some help but his suggestions will be very approximate at best.
Modeling Operations Relation objects are not required in a model. However, if the modeler wants the benefits of the validation utility , the model must be able to pass the closure test, and in order to pass the closure test, entity relations must be manifested. Manifestation is needed because the system cannot be allowed to determine the relations. What might appear to CoreModel to be a valid relationship could be an illusion created by object attributes, and that could leave some model entities with no supported valid relationships. Also, the modeler may want to manifest all implicit relations just to assure himself that his model is valid. CoreModel's relation manifestation utility assists the modeler with the manifestation of relations through their actualization as relation objects. Operation
The objective of the relation utility is to assist with the construction of relation objects by identifying unmanifested relations and by validating existing relation objects. For a model to adequately benefit from the operation, it must be as complete as possible. Due to the complexity of the subject and concepts, using this tool tends to be iterative. After running and acting on its recommendations, the model validation process should be run and its suggestions should be acted upon. If necessary, the relation manifestation process should then be re-run and its findings again reviewed. Again, the validation process may be re-run. This process should be repeated until the system reports a successful closure. The model is unchanged by these operations; after some thought, it was decided that an automated system cannot accurately and reliably create relation objects. Therefore, the system recommends and it is up to the modeler to make or ignore any recommended changes. The utility process is complex and resource intensive. For a large model on a slow computer, or for a highly normalized database, it may run for a very long time. A four hundred megahertz computer requires hours to evaluate the demonstration CIA World Factbook model and a three gigahertz machine requires nearly an hour. If the model is being created by a team, the modeler should insure that nobody else is working on the model during the process. For the report, CoreModel posits three groups of relations:
The process begins by evaluating the validity of relation objects that have been created by the modeler. The next step is a search for all implicit relations in the model. In this, CoreModel may or may not be correct because he must rely entirely upon what is thus far recorded in the model, so the modeler must be careful to review the system's findings. ( Beginning modelers, please note: This step illustrates one of many reasons for consistency in modeling. A beginning modeler should strive for a consistency in his operations that covers years as well as projects. ) The system will not duplicate a relation path in the second step; i.e., it will not specify multiple relations between two entities. Once it has established that two entities are related, it moves on. The problem with this automation is that the identified relationship may be a tenuous or eufunctional one which then obviates the expression of a more desirable relationship, or it may mask the fact that there is no truly adequate relationship. The modeler must be alert for this problem because it requires judgement calls that can be made only by a person with a knowledge of the data. Finally the system identifies any unmanifested relations. Unmanifested relations are all of those that were identified as implicit relationships and which have not been manifested in relation objects. After the process is complete, CoreModel gives the modeler a complete segmented report of the operation. The report is designed to assist the modeler in creating relation objects that may have been overlooked, and can be left on screen for information while objects are being created or edited. ( Many new modelers do not understand the relational construct and the managers of a very large software company add to the confusion by intentionally encouraging irrelevant and obfuscating hard-coded relationships which adulterates the relational construct. The situation is worsened by the fact that the monopolistic position of that company makes it the only source of information for many junior level developers. To draw attention to that dismal situation and to insure that CoreModel does not participate in promoting such ignorance, he will not instantiate relation objects. In other words, the only valid manifestation of a relation is a user-generated SQL statement. )
Modeling Operations CoreModel will test the validity of a model upon demand. Model validation is a general evaluation of various aspects of the model to ascertain its current degree of validity. It is validated simultaneously against both the conceptual and mechanical realms. The use of this utility is recommended before changing abstraction levels and its use is highly recommended prior to delivery. Although it cannot guarantee a valid model, it can certainly catch many mistakes. Operation
The validation process has no effect on the model or on any of its supporting data. The sole purpose of the process is to report possible problems to the modeler so the modeler can correct and improve the model. The modeler may ignore the report or even ignore the utility entirely and proceed with model development and deployment. It is not unusual for the report to show numerous errors in a model. The use of this tool should be iterative because the correction of a model's errors can create or uncover other errors. If working with a team, the modeler should be certain that nobody else is working on the model when the process is started. The suite of tests that is selected by the validation module to run against a model is dependent upon the model abstraction level and the current state of the model. At this time, the validation process will not check the specified model type or RDBMS characteristics, although those features are planned for the future. The process begins by compiling simple statistics on the model for two purposes. The first is to give the modeler a hard copy report on them, and secondly to insure that the model contains minimal object requirements. If there are no columns, for example, the process notes that fact and shuts down. Table and column relations are checked next. This identifies such problems as a table with no columns or a column with no table. This type of error is easy to make when working on a large model. Column characteristics are checked. For example, a missing data type or a text type of data without a size will be flagged. Since the operation is dependent upon the relation objects that the modeler previously created, the relations are themselves closely scrutininzed next. Each will be matched to its tables and columns and the characteristics of the objects will be compared to insure that a valid relation is possible. Indices are checked and then all of the keys are closely scrutinized. Every table is checked for a key and the keys are matched to the column objects which are then checked for validity as a key. CoreModel is highly suspicious of a table that cannot be keyed. After all other tests are run, closure is tested as described in the following section.
Modeling Operations The closure test provides additional validation of a model. It is now part of the validation test suite. When time and resources permit, it may be broken out into its own test because of its unique abstract quality, but its extreme dependence upon the completion of the model makes removing it from that suite costly. Operation
As was stated in the previous section, this test has no effect on the model or on any of its supporting data. The sole purpose of the process is to test the validity of the model and report to the modeler. The modeler may ignore the report or even ignore the utility entirely and proceed with model development and deployment. Before proceeding, it should be stated that closure is a CoreModel concept. It has never before been explicitly stated as a part of or a requirement of the relational model concept. The modeler should be aware that closure is not, at this time, part of mainstream modeling in spite of its usefulness and theoretical basis. Closure cannot be made until the model has been thoroughly validated. Closure cannot be achieved in an invalid or incomplete model. CoreModel checks the relational validity of the entire model by constructing a table chain to check for closure. If any elements of the model cannot be included in a single chain, the model is fragmented and cannot meet the specifications of the relational construct. Closure failure is not a universal invalidation. For example, by design and intent, the CoreModel system has tables which are fragmented from its main chain. However, if closure cannot be made, it must be the modeler's intent, or the model is invalid without exception; without qualification. The closure report prints the results of the test and prints a copy of the chain that was constructed during the test. The closure test chain may be of any degree of complexity. CoreModel will trace all loops, multiple branches, dead ends, backtraces, etc. As long as CoreModel can construct a single unfragmented chain of any degree of complexity for the model, the model will pass the closure test. The complexity of some models transcends all practical consideration. Therefore, CoreModel uses brute force to construct chains. To attempt sophisticated constructions would leave the system vulnerable to unforeseen overload by models of great complexity.
Modeling Operations Normalization has been considered for inclusion within the CoreModel utilities. Some thought indicates that a system can normalize a model only if the model is first adequately prepared by the modeler to make it possible for the system to identify normalization opportunities. That preparation would require as much time and work by the modeler as would normalization. It might be argued that the utility would assist those modelers who do not understand normalization. However, the preparation of the model for system normalization would require that the modeler thoroughly understand normalization. Therefore, CoreModel does not normalize.
Modeling Operations This operation is an attempt by CoreModel to model the source, so it is not an import in the usual sense. Directing CoreModel to perform an import tells him to establish communications with the source system, request permission to read specified files, and begin reading the information. Then, using that information, he attempts to create appropriate model objects, making them as complete as possible. Before an import operation can be performed, the system administrator must first create a model object into which the import will be made. The model must be at either the physical level or the implementation level because only those levels can import. The administrator must also create a data connection object which will be used by the modeler to connect to the source. Connection testing is accomplished by simply pressing the test button, so it should be tested before it is released to the modeler, and then should be tested on the modeler's computer. Four types of connections can be made to the target for an import. ODBC, OLEDB, DML, or CDO. A DML connection directs CoreModel to a DML file. A CDO connection is for AxleBase. ( DML means Data Modeling Language. DDL means Data Definition Language. Since most of the industry uses the DDL term to also refer to DML, thought is being given to changing the CoreModel nomenclature. ) Because an import requires a data connection, the modeler should consider reading the CoreReader documentation at http://CoreReader.com to learn some of the ways in which ODBC and OLEDB data connections and various data sources change the characteristics of the import operation. Because of the tremendous differences, it is impossible to describe the results of all imports in this document. The modeler may consider using a combination of imports. For example, using an ODBC connection for tables and columns, and OLEDB for the other objects sometimes yields best results. However, the result is also dependent on the capabilities of the database server. Operation:
The drop down list of connection names show the connections which have been set up by the CoreModel system administrator. Each connection name corresponds to a connection to a specific database under the control of a specific database manager on a specific computer. Select a connection and test it by pressing the test button. If CoreModel cannot connect to the source, the modeler should notify his CoreModel system administrator. The import screen has a checkbox for each element type. Checking a box directs CoreModel to read the specified type of elements to create the corresponding CoreModel objects. Checking all of them will read all that are available. The elements are:
Again, not all database servers and connections can deliver all of the objects. If an object cannot be delivered, CoreModel will make a note in the operation report. The process distinction between keys and indices can be a bit confusing because they are not handled uniformly by all database servers. The keys selection will certainly import only the keys. The index selection may or may not also import keys. If both are selected, CoreModel will not allow duplications and will simply report attempted duplications. Therefore, it may sometimes be best to attempt to import both. It may be best to do the imports before beginning work on the model to avoid name conflict errors between created objects and elements that are being imported. There is much sloppiness in database managers that are now on the market. They allow every kind of character in object names including spaces. Some send extraneous system data. Some fail to send desired data. Because of this sloppy situation among database managers, most garbage collection was removed from the utility. The modeler must be prepared to clean, purge, and expand after an import. CoreModel will attempt to run an import operation through all errors to the end of the process. He will get as much of the data as possible. This behavior is an attempt to simplify operations as much as possible regardless of their internal complexity. If the import operation results in a bigger mess than the modeler cares to clean up, all objects in the model can be dropped by the administrator with the push of a button. The import report of the operation will list all of the objects that were imported into the model. The report will also contain descriptions of any errors that may have occurred. Each imported object will contain in its comment field a notation of the fact that it was imported. This is a reminder from CoreModel to the modeler. If the modeler has not removed the note from an object, then he may have forgotten to review that particular object after the import operation. In today's environment where security is a pervasive stumbling block, security presents a problem for systems that must communicate, so the system administrator and the modeler should identify and nullify all security flags to insure that they will be bypassed in the operation. All security levels for all objects must be checked. For example, if the particular data connection is not made with a logon id which has access to all of the tables, columns, etc., then some tables may not be imported in their entirety. Although he is not an expert on the subject, CoreModel understands the SQL language well enough to do a fairly good job of importing from a DML file. If the SQL is valid, the error rate will usually be lower from a DML import than from an attempt to obtain information directly from another system. A DML file allows the modeler to alter and enhance the information before the import operation to obtain a more detailed and accurate model. If errors occur, the modeler can correct the DML, drop the model, and redo the import. After it is created, a DML file bypasses security and Database Administrators and the modeler can work freely. This alone can increase the modeler's efficiency. Considering all factors, if the modeler has a choice of import sources, the DML source may be far more effective than a direct connection. DML files must use standard terminators. CoreModel recognizes both the semicolon and the keyword GO.
Modeling Operations Unlike the import utility, this utility actually does an ordinary export operation. It is entirely separate from the CoreModel Metadata Server and has nothing to do with model instantiation. A modeler may export any or all of his data at any time. When an object is exported, only the top level object is exported without its components. It is not possible for the structure of an object to be maintained in an export, so only information is exported and any structure must be inferred. The system allows an export into text files or web pages. (Since the complexity of EDI is beyond most people, it has been deferred. Requests for it will be considered.) Operation
Multiple objects may be selected for a single export up to and including all of the information in a model. Output files will be assigned the names of the objects that are selected for export. The export will go to the current system default directory which is displayed on the export screen unless it is changed by the operator. If a path is entered that does not exist, the system will try to create it. A file name must not be entered. If text output is selected, a text file will be created which can be red by any word processor. If the output is to html, the system will create standard web pages with a page for each type of objects. The system allows the name of a home page to be entered which will cause the creation of a home page with links to the other pages. This documentation is included in the utility's list of things that can be exported. This makes it possible for everybody to have printed documentation. This is permitted under the license terms as long as the copyright is preserved and the license terms are followed. For example, the documentation must be complete at all times, and the copyright and license must be present.
Modeling Operations The modeler may actualize a model by delivering a DML script to the database administrator, who will run it within the RDBMS, or circumstances may require bypassing DML to affect direct actualization. CoreModel supports both of those methods. CoreModel can instantiate directly into a server if the server is capable of understanding industry standard SQL. CoreModel generates ANSI 92 compliant SQL DML. ( Since those who participate in this sport usually stand above the average bear, there is no need to degrade the vocabulary. However, one might legitimately question the use of the word instantiate if a simpler one would suffice. Simply put, the word instantiate is the best fit for a process of model actualization which can be performed multiple times in multiple locations on multiple platforms at the whim of the modeler; a task for which CoreModel was designed. ) In the implementation module is a button which loads the instantiation form. On that form is a field for the model name, which has been filled in by the system, and a drop down list of connection names. Operation
An operation report will be generated listing all objects created and any errors. As usual, the report is saved in the output directory and displayed on screen. If a particularly massive model generates a report larger than CoreModel can display, a word processor can be used to view it. A valid model is not a requirement for an attempted instantiation. If an error in an object precludes its instantiation, the object will be bypassed and the error will be noted in the report. If there are basic errors in the model, the instance may fail, and CoreModel will attempt to find and report the reason. If errors are encountered in the process, it is recommended that the modeler first correct the reported errors and then consider the validity of the remainder of his model before making another attempt. Sometimes, a single error in a model can be indicative of other potential problems. Similarly, the modeler must be knowledgeable of the various target platforms. For example, desktop database managers typically cannot support the actualization of views. However, an attempted instantiation in an invalid target should do no harm other than perhaps create some trash to be cleaned up in the target. If the modeler employs a strange nameing convention, then the burden of handling those strange names must fall upon the modeler's shoulders, and he must somehow delimit or identify the non-standard names to the target in the manner that the target requires. For example, some beginning modelers actually embed spaces in object names. Direct instantiation is extremely fast and convenient. Pressing the button and seeing the finished process report pop up furnishes a rewarding treat for those who put so much thought and time into models. However, indirect instantiation via DML is strongly recommended because it affords great flexibility and allows enhancement of the modeler's work. Since the process is so easy, a combination might also be adopted with attempted direct instantiations to identify problem areas followed, perhaps, by the real thing through enhanced DML. Instantiation ordinality is hard coded. The flow is usually tables, secondary indices, views, and processes. If that presents a problem for the modeler on a project, the enhanced DML instantiation approach is provided. The drop down list of connection names shows the connections which have been created by the CoreModel system administrator. Each connection name corresponds to a connection to a specific database under the control of a specific database manager on a specific computer, and he will also set up one or more for the location of DML scripts. A connection to the target will be established by CoreModel when the do-it button is pressed. If the connection cannot be established, CoreModel will stop and pop up a notice. There is a test button on the form. It allows the modeler to test the connection before running the process. All security issues must be addressed before an instantiation attempt. For example, if the data connection is not made with a logon id which has table creation privileges in the target database, then the tables will not be created. That is only one example, and the modeler and system administrator must be vigilant in this area. If the objects already exist in the target database, CoreModel will attempt to over-write them, regardless of whether or not they contain data, so the modeler should be sure that he is connected to the correct database. This behavior is by design to allow the modeler to make multiple attempts without worrying about target preparation. If the database manager is AxleBase and if the database does not exist, CoreModel will ask the modeler if it should be created. If the answer is affirmative, CoreModel will create the new database. AxleBase is the only database manager with which this can be done.
Modeling Operations Each model is provided with a schematization utility which will print a rudimentary model schema for reporting purposes. Operation
Modeling Operations CoreModel will assist with an implementation size forecast based upon what is known of the model after the model is sufficiently developed. CoreModel will use whatever is available, but if the model is insufficiently developed to entirely preclude a forecast, the modeler will be notified. Operation
The forecast report presents the total size, the sizes of the elements that went into the total, elements that could not be totaled due to an insufficiently developed model, errors in the model, and any errors that were encountered by the utility. The report will go into a disk file for editing and distribution and will be displayed on screen if the modeler so wishes. Size projections do not account for the vicissitudes of the various brands of database managers. For example, Ms. Access and some servers sometimes consume considerable additional disk space. Sizes cannot exceed the petabyte range as of this writing. Indices are treated separately on the report to provide more detailed information.
Modeling Operations If the system administrator installs the CoreReader module with CoreModel, it will give the modelers a means of querying any database in any data source. It has been used for everything from mainframes to spreadsheets. CoreReader lets the modeler quickly query databases by simply pointing and clicking.
Modeling Operations As modelers, we enjoy wall charts of our models as concrete evidence of our work. What long-time modeler has not papered the walls of an office with overwhelmingly complex drawings? And we sometimes enjoy looking at charts on the computer screen. Now, let us consider the reality of graphics. When working on a small model, the modeler finds that he knows the model so well that looking at a pictorial representation of it gives little help in real modeling work. We usually find ourselves creating a small model and putting it into production without even considering a graphical display. When viewing a graphical display of a large model, where help would be appreciated, the modeler quickly becomes lost in the fascinating myriad details on the screen, and then discovers that he is thoroughly overwhelmed visually and receives little real help from the picture. A picture of a large model is little more than an impressive picture. The uniformed might then think that a picture might be helpful with specific information. However, those of us who have spent time with such things, know that when a modeler needs to look up a specific piece of information, he might as well look in organized lists and tables, where computer systems are imminently capable of assisting him. So will we then totally discount graphical displays of models, and refuse to consider them? Absolutely not. As pointed out, they are personally appreciated by the professional who invests his life in creating models. But the actual work of modeling is far more important to the modeler than pretty pictures. Therefore, a graphical display will be added to CoreModel sometime in the future, but it has a very low priority. In the meantime, some who have become accustomed to the old fashioned "gui" interface may complain. But from the beginning, CoreModel was intended to be aesthetically pleasing on an intellectual level because its function is the no-nonsense creation and management of models of reality.
_________________________ The metadata repository is extracted from the models by the system, and is presented in an external interface for organizational use outside of CoreModel. To preclude unnecessary ties to specific tools and vendors, the repository interface attempts as much abstraction as possible, but entity integrity must be maintained, so the model is presented as an abstract entity. Operation
After a modeler feels that his model is ready, he can make it available for publication in the data dictionary by pressing the button. That action releases the model to the system administrator for publication. The system administrator can then cause the system to publish the model elements as defined metadata. The link between the repository and the model is dynamic, so that after publication, any changes that the modeler makes to the model will be reflected in the repository interface. IMPORTANT! The modeler must be aware that the repository interface is designed to allow external tools to tie directly into it as metadata consumers, which means that other systems in an organization may become linked into the CoreModel model after the decision is made to publish. For that reason, after a model is released to the Metadata Server, only the administrator can rescind the publication.
_________________________ After studying the previous sections, the professional should have an understanding of the mechanical and architectural aspects of CoreModel and be ready to apply the tools to a project. However, a new tool can take some time to fit into the craftsman's hand, so this section is intended to reduce that time by covering various scenarios where the modeler is given a specific task. The world of modeling reflects the variety and inconsistency of the reality that we are asked to model, which increases the complexity of modeling projects and produces a high probability of confusion about the requirements of a project. Modelers are frequently handed tasks which are well thought out by management, but which require careful tactical assessment by the modeler before starting. More than anything else, the modeler should pause to assess requirements to insure that he recognizes an identifiable objective before beginning his task. After the objective is identified, the modeler can then more easily map the work into the CoreModel tool; i.e., the hammer can be swung at the correct nail. CoreModel is architected to flexibly address various modeling objectives in numerous ways. Following are some possible requirements with general guidance in working on them. These are not intended to be detailed instructions for the beginner, but merely coaching to help the modeler get started.
Modeling Operations At times, the modeler is required to pursue a modeling task from concept to instantiation, so CoreModel will carry the task through to this goal on various platforms and through various methodologies. It supports multiple instantiation. The first task will be the creation of the model object. For this, the modeler will sit down with the system administrator, who will create the model to the modeler's specification in the administrative module. The model will be created at the conceptual level. Returning to his workstation, the modeler loads CoreModel and finds the new model listed on the main menu. He double clicks on the name and the new model is opened. He first opens the create screen for entities by highlighting the entities and pressing the create button. He creates all of the entities and then does the same for the attributes. This may be done quickly or it could be a large project requiring days. CoreModel does not require perfection. The purpose of the conceptual level is to bring the model as close to the next phase as possible. When he is satisfied with the work, the modeler presses the step button to move the model into the logical level. Resuming work at the logical level, the modeler finds that his entities and attributes have been translated. This would be a good time for the modeler to consider any domains that will be used repeatedly in tables. Details are then added to the column objects. Since the modeler will want to use the validation utility, he also begins creating relationship objects. When the model meets his satisfaction, he moves it to the physical level by pressing the step button. At the physical level, more details are needed. Also, the required index objects will be created and miscellaneous constraint objects may be created. The model is beginning to take shape. The standardization utility may be run repeatedly by the modeler to preclude mistakes in his critical object names. The validation utility also might be run numerous times to assist. The model is then moved to the implementation level. At the implementation level, the model is close to completion. Here is where the finishing details are applied. When all seems to be ready, the modeler asks the system administrator to create a connection object for his instantiation. Experienced modelers and developers know that the instantiation will invariably have problems and may need to be repeated, especially for large models. When all seems to be ready, the instant button is pressed to load the instantiation form, the proper connection is selected, and the do-it button is pressed. After instantiation is complete, CoreModel's export utility might be used to begin the creation of a formal documentation of the model.
Modeling Operations Sometimes the modeler is required to make a model delivery via hard copy. CoreModel will create that kind of deliverable, and will do it in detail with supporting documents, and will do it from any level of abstraction; conceptual, logical, physical, or implementation. A project that requires a hard copy deliverable may specify a logical, physical, or implementation level. The model will be worked on as in the previous scenario until it is adequate in the specified level. Various reports can then be produced that the modeler will use as the core of a formal delivery document. The export utility will create a list of the model contents. The schema utility will provide a skeletal organizational view. The validation utility might be used to document the legitimacy of the deliverable. Finally, the model might be instantiated in the form of a DML document. All of those things would then be molded into the deliverable by the modeler.
Modeling Operations There are times when a manager, modeler, or administrator has a set of functioning systems in production and wants to inspect them for flaws, maintenance, or changes. CoreModel provides as non-intrusive comprehensive means of doing that with no disruption of the living systems. CoreModel will extract required information, abstract the results into a model, and then provide the tools to study it. To meet this type of scenario, the system administrator will again create a model object, but this time, the object will conform to the characteristics of an existing system. Of course, it could be created at any level, but since it is a production system, the appropriate level at which to create the model is the implementation level. At the same time, the administrator will also create a connection object to allow the modeler to connect to the external system. If a connection cannot be made directly to the functioning system, then the connection will be to DML file. The modeler will contact whomever administers the functioning system and request that industry standard DML be generated. He will direct CoreModel create the model objects from the file after he presses the import button. The modeler then loads the model, and imports the external system. The import will be imperfect, so the modeler will flesh out any required information afterwards. The model of the living object is then available for study by the modeler and documents may be produced from it for study by management.
Modeling Operations A modeler may be called upon to provide metadata support for an organization. In that case, CoreModel provides tools for capturing the required data and automatically performs all abstraction transforms. Finally, it provides publication tools, and even supports autonomous 24x7 automated dictionary updates via its internal Metadata Server. The Metadata Server also includes a web site administrator for those modelers who need to place metadata on a net. This scenario might proceed like the previous ones where a model is built or an existing system is imported. But additionally, the modeler will publish his model when it is ready. After publication, he will verbally tell the administrator of the action. The administrator will make it available to the Metadata Server. CoreModel's Metadata Server will translate the model into metadata and thereafter keep the model published for whomever and whatever needs the information. The negative aspect of publication is that after doing it, the modeler cannot undo it, so he should complete and prove the model through instantiation before publication. However, after publication, the modeler can continue to maintain the model, so his changes will be dynamically reflected by the Metadata Server.
Modeling Operations A modeler may be required to perform a combination of any of the above tasks. CoreModel was architected for flexibility to allow it to support a combination of those tasks and to support subsets of those tasks. Regardless of the deliverable, CoreModel is designed to provide adequate management reporting support during and after completion of a project. ( Note: For small projects, or when we are rushed is rushed, which is usually the case in this profession, we frequently skip the preliminary work at higher abstraction levels for the sake of efficiency, but those who think that the higher levels should be consistently skipped to do all of the work at the implementation level should, perhaps, emphasize database administration and leave data modeling in the hands of data modelers. )
__________________________________________________ Chapter Advanced Modeling __________________________________________________
_________________________ Because our work is fun. The Advanced Modeling Topics section is oriented toward the Master Modeler; the modeler who has addressed his work as a profession. Others may find it frustrating and of little or no use. CoreModel demonstrates the direction that modeling will take in the twenty-first century. It does that by providing real tools which are intentionally designed to reflect underlying theory for studying and manipulating the concept that is referred to as "model". It is believed that additional controlled abstraction beyond the present levels is possible, and not only interesting in its own right, but potentially extremely rewarding on practical levels within the domain of high end models. Even if the thrust of the CoreModel project were not so much fun, its practical potential would make it valuable. The advanced structural characteristics of the CoreModel design, and the fact that CoreModel is free of the control of major corporate interests, provide an ideal environment for addressing advanced concepts in modeling which have been undreamed of in older tools.
_________________________ The following sections address interesting theoretical topics. These are not at all needed by the modeler who is trying to earn a living, but they do make the work far more interesting, and could be worthy in their own right of research by those with the time and telent. But possibly there are hidden practical aspects of this section. In all human endeavor, it is the man who can grasp a bit of the next higher level of his job who best understands the level in which he works on a daily basis, and it is the man who finds his job interesting enough to draw his attention into higher levels who does his job well. Making it even more fun is the fact that these topics are not the defining end of modeling theory. But these and others on their level must be addressed and nailed down before the next levels of theory can even be adequately formulated.
Advanced Modeling Of particular interest to the Master Modeler is the fact that CoreModel is the result of the solidification of theory. The design of CoreReader began by outlining theory on paper and then methods were developed for expressing the theory in the real world. Finally, a user interface was developed which expressed that work. During practical work, that fact may not be obvious because CoreReader's user interface simplifies the result. CoreModel's copyrighted approach to modeling made theory its foundation. This insures that its mechanical structure will adapt to unforeseen real-world problems. The operational mechanics that the modeler employs within it are actualizations of the theoretical realm. In addition to the static concepts of table, row, etc., there are dynamic concepts within the modeling construct. Both are addressed on behalf of the modeler. The workflow is manifested as modules through which the model is moved from inception to completion. The expression of workflow tasks is addressed by first isolating the taxonomic from the analytic. Since the taxonomic and the analytic are entwined in interdependence, they are first identified and then isolated so that each can be addressed most efficaciously. This approach resolves exceedingly complex tasks into relatively simple mechanical operations. Rather than engendering boredom in the Master Modeler, this relieves him of those burdens so that he can address higher level concerns. ( A graphical map can be seen on the internal architecture page.) There is a main module and five work modules which are loaded from the main module. Four of the modules are actualizations of the algorithmic constructs of the four hierarchical abstraction levels of the modeling effort. They are conceptual, logical, physical, and implementation. It is in those four modules that the modeler does his work. The modeler moves his model to the next abstraction level as his work progresses. So much of a modeler's work takes place as intellectual concepts that CoreModel is strictly architected to maintain strong conceptual images at all times. However, although it maintains strong conceptual images, CoreModel is intentionally designed with canonical division permeability to support the modeler's creativity. The Master Modeler will notice that he is always aware of the current location of his model while maintaining the freedom to move about in the theoretical realm when the need arises. He is even allowed to address his model at incorrect abstraction levels if he consciously decides to do so. The administration module sits to the side of the four modeling modules. It provides support for the many complex needs of the modeler so that he can devote his thought to creative modeling.
Advanced Modeling When I have time. This is a place holder.
Advanced Modeling The closure chain itself could be the subject of an entire dissertation. There are, of course, complex mathematical characteristics within it that are worthy of investigation for large models as academic exercises. However, practical promises of a manifested chain are mostly lost in its serious complexity. The chain is presented in the final report because it might be of interest to the modeler, but for years, the utility only reported success or failure at closure due to the chain's lack of utility. A strictly theoretical approach to closure, although offering little practical benefit, can be extremely interesting to a modeler and may sometimes give insight into hidden characteristics of complex models. This approach transforms closure from the verb of the practical endeavor to an adjectival noun in the abstraction. Closure in this case may be seen as a postulated characteristic of a model, and the effort becomes the investigation of the postulated continuity of a topological entity. If the entity is a topological space, one might address the validity of the space as a function of closure, with the ultimate objective of mathematically stating the validity of the model. Failure to prove the continuity of the entity is a failure to demonstrate closure. At the practical level, that failure to demonstrate closure becomes a disproof, but at the theoretical level, the failure is merely the failure to prove. Therefore, for large and complex models, failure to demonstrate closure through CoreModel's mechanical means does not prove the lack of model closure; such proof, if found at all, could only be done mathematically. We might suspect, then, and therefore postulate, that the existance of model closure, whether proven or not, reveals the existance of a topological space which was created by the manifestation of the model. As such, the model becomes subject to extensive topological investigation and analysis. One question that might be asked is whether the chain exists as an artifact within the space, or if the convolutions of the chain have shaped the characteristics of the space.
Advanced Modeling It is not possible for a valid model to contain multiple closed chains. If such is discovered, then the model actually contains multiple databases. For example, the physical architecture of Microsoft's databases presents a system database with the primary database. There is nothing wrong with that, but it sometimes leads young modelers to think of the two as one. In actuality, they must be modeled separately regardless of how the big name brand's store them. In other words, there is not such thing as an asymptotic relationship. The manifestation of asymptotic chains in a defined topological space merely means that the space was defined to contain them. But in the relational construct, there is no relationship. An interesting exercise might be to investigate the construct approaches within the space. Such investigation might reveal latent closure tendencies or lack thereof. For example, in the cited Microsoft example, the model closures may not be at all asymptotic despite their storage method.
Advanced Modeling One may notice that the modeling mechanisms do not address the question of completeness. Does the model contain all that it needs? The reason for that is direct: Only God's universal model is complete. All others are subsets and derivations of the universal model. We treat them as complete so that we can get on with matters at the human level. For example, one might create a huge model of a nursing home, find that it is valid and viable, and put it into production with great success. A year later, managment may realize that they forgot to tell the modeler to include extended families. That oversight does not make the model invalid. And that is one of the reasons for CoreModel's tremendous flexibility. The modeler can re-enter the model at a low level and begin building the new entities into it up through the production level. If external systems have been consuming the model, the new objects will automatically be made available to them.
Advanced Modeling When I have time. This subject will be approached with trepidation because I am not a mathematician and certainly not a topologist. However, this section cries out with an obvious need for expression, so it will be done as soon as I have the time. This section promises to be a whole lot of fun.
_________________________ Following are brief descriptions of planned additions to the system. Although based in theory, these are planned to be valuable real-world additions to the modeler's toolbox. These tools will be of particular interest to the Master Modeler on staff and to technologically aware management. It is no longer a secret that CoreModel and other projects are the work of one man, so they do not move rapidly. These additions were targeted for construction in 2002 and 2003, but have been pushed back several years by the CoreReader and AxleBase projects.
Advanced Modeling This is now envisioned as an on-demand utility which operates as one of the modeler's utilities to quickly generate a coefficient of complexity for any model for inclusion within larger system models and for management studies and evaluation. This is envisioned as a high-end utility for generalized management guidance. Its purpose will first be to provide a feasibility assessment tool for projects that may be too complex for human evaluation with hidden levels of complexity. Then, it will provide resource planning assistance for both hardware and operations personnel.
Advanced Modeling The current tentative plan calls for an on-demand utility which quickly generates isomorphism evaluation reports for management planning.
Advanced Modeling This is an excellent concept that was developed in the biological sciences, and is borrowed by CoreModel because it is imminently applicable to modeling management. The CoreModel infrastructure provides an excellent means of addressing model phenotypic plasticity. It is now felt that the best way to address the issue would be through an on-demand utility that will generate a phenotypic plasticity gradient. Currently, I foresee this utility as having limited use, but being of great value when evaluation is needed by an organization.
__________________________________________________ Chapter System Administration __________________________________________________
_________________________ If you received the CoreModel system from a friend, be aware that you can find news and updates on the web at http://www.coremodel.com/ . Send mail to administrator@coremodel.com. In a multi-user installation, it is recommended that an individual be appointed to be the CoreModel system administrator. The system administrator may be responsible for maintaining all of the system wide data tables, maintaining security, supervising the Metadata Server, performing backups, providing management direction, and providing general support for the modelers as outlined in the following sections. For large installations, certification of the administrator is recommended.
_________________________
System Administration To install CoreModel, place the files in a temporary directory and run the setup.exe program. After it finishes, load CoreModel. The first time that CoreModel is loaded after installation, it will notify you that it is completing the installation. CoreModel must be installed on each computer from which it will be used. It installs from a standard setup exe. As part of the installation process, it builds and initializes a local database in the same location as the exe. After it is installed and initialized, CoreModel may then be redirected to a central database for multi-user and enterprise level operation. The system activity log will contain records of the initialization. If the installation is interrupted, assume that the installation is corrupted, uninstall, delete all files, and restart. When CoreModel is first started, it creates a settings file in the same directory as its exe. That settings file tells it where its database is located. CoreModel may be redirected to a new database by unloading the system and then changing the database location in the settings file. If you mess up the settings file and you are not familiar with database settings, the easiest way to fix it is to delete it and start CoreModel to create a new one. If CoreModel can find neither a settings file nor a database when it starts, it creates a default settings file, warns the operator, and shuts down. The new settings file can then be altered. Note: Each time that CoreModel awakens, he checks the integrity of his database. To create a multi-user installation:
It is critical that the system administrator of a large installation be aware before beginning that CoreModel contains various servers which function autonomously when placed in operation. The establishment of a CoreModel installation should include server deployment planning as an installation architecture. For an individual using CoreModel, a single computer should suffice for all needs. For multiple users and especially if unattended servers are employed, the following might be considered as a minimal configuration where the database is handled by a database server.
System Administration CoreModel will not install over the top of an old installation. The old one must be uninstalled before installing the upgrade. In a multi-user installation, insure that all instances are shut down before beginning the upgrade. Consider locking the database before proceeding. Upgrade all instances before allowing anybody to log in. Uninstalling will destroy your database. Also, upgrades are not always compatible with old databases. Then begin the upgrade.
If the upgrade is found to have an unacceptable problem, roll back to the previous release and restore your old database. Some previous releases are available in the download archives for rollback.
_________________________ The bad news is that the administrator who creates a CoreModel installation will have his hands full with over sixty system parameters to maintain and possibly thousands of records to be entered into sixteen system tables. The good news is that the installation procedures assist the new administrator by creating hundreds of records in the tables and setting default parameters. For a single user operation, the initialization should suffice until the user wants to enhance his productivity through the tools provided in the administrator module. However, for a multi-user operation, and especially for an enterprise level operation, the administrator should immediately begin reviewing, validating, and updating all system parameters and all system tables. After initialization is complete, the main system screen will be displayed. Pressing the admin button will load the administration module which contains all of the administrator's controls. The system administrator may log on to any CoreModel workstation or server and may open the administration module on any of them. In the administration module, data may be added, edited, or browsed by selecting the data set name in the window, and pressing the appropriate button on the left side of the admin form. For example, click on domains in the window to highlight it and press the create button. The domain create screen will be displayed. Multiple objects may be highlighted by holding down the control key while each one is selected. Then pressing the create button will display the create forms for all of the selected objects. Drag the cursor down the window will also highlight multiple objects. The system is designed to allow the operator to display data as he needs it. However, CoreModel will close the create screen when the edit screen is opened to avoid confusion. On the right side of the form are buttons that control the various aspects of the system and perform system maintenance. When a button is pressed, it will identify itself, sometimes give detailed instructions, and provide a chance to back out. Regardless of whether or not the security will be used, the administrator should make himself and other users known to the system by creating a personnel object for each. The administrator's personnel object must immediately be given administrative access to the system. Next, model master records that may be needed by the modelers to begin work should be created. Finally, some time and thought should be given to planning the population of the various master data tables that are described elsewhere in the documentation, so the modelers will have the data that they need to do their work. This last step will be a never ending job for the administrator. As noted later, the same three types of data maintenance screens that are described in the modeling section for the modelers are also used by the administrator. A subsequent section will describe model management and how to access the data maintenance screens for models. To maintain consistency and ease of use, the data screens for models may also be accessed like other create and edit screens on the admin screen.
_________________________ The CoreModel Database Backend CoreModel's Database The installation includes an AxleBase database manager. When CoreModel is installed, AxleBase installs with it. CoreModel handles the interface to AxleBase. If desired, extensive documentation of the AxleBase database manager is available.
System Administration CoreModel looks around for his database each time that he wakes. If he detects an empty database when he is started, he will begin initialization of it. The database initialization must not be interrupted. Interruption could corrupt the installation. He notifies of initialization completion. The default database will be found under the CoreModel location in a directory named \db. It is a standard AxleBase database. As part of the initialization, the system assists the administrator by creating numerous records in the master tables such as relation types, data types, etc. After he initializes them, he doesn't care what is done with them. Those tables should be altered to fit local needs. He also sets initial values that he likes in the system settings, which should be updated in the configuration utility. When CoreModel is upgraded, if he finds an old database in use, he sometims requests permission to create or update tables. In that situation, either he must be allowed to make the change, or he will unload to allow the upgrade to be rolled back.
System Administration There is a backup utility that can be run from the administration panel by pressing its button. It will run only if a backup location has been entered into the system control parameters. Frequent backups should be made. All databases become corrupted sooner or later. Daily backups are minimal. If the database becomes corrupted, stop all processes, exit all modelers and administrators, and press the backup button. When the backup starts, CoreModel tells AxleBase to lock the database until he is finished. (This obviously applies only to the AxleBase database that ships with CoreModel and not to a database server. )
System Administration All databases accumulate empty space during use, and eventually may become uncomfortably large. A utility is included in CoreModel specifically for packing the database to eliminate wasted space. To use it, open the administration module, press the pack button, and follow CoreModel's directions. The pack includes additional AxleBase utilities to purge and tune the database. When it is run, it :
In a multi-user environment, all other users must exit before the packing begins. None of the CoreModel servers can be running. The database may be corrupted if any other person or process is using it when the packing begins. At the beginning of the operation, AxleBase places an exclusive lock on the database. That prevents anybody else from opening the system. Before packing, adequate backups should be made and verified. CoreModel makes a backup before starting. If the backup fails, CoreModel will announce a failure and may announce that he will unload to abort the operation. The database should then be restored from a backup. After the packing is concluded, CoreModel will announce a successful pack. Packing can change things in the database that CoreModel may have loaded. Therefore, he should be restarted, and the data should be checked. If there is a problem, it is usually immediately obvious.
System Administration CoreModel was first built with the intent to allow it to be moved from the Ms. Access database to a database server. It has all of the functionality needed by a server. But since it is now using the AxleBase database, that is no longer necessary and that functionality may be removed eventually. For database connection parameters and instructions on connecting CoreModel to a server database, see the database settings in the system settings section.
_________________________ A model is an object in CoreModel. Only the system administrator can create, replicate, and destroy the model objects. When preparing to create a new model object or to perform an operation on an existing model, the administrator must confer with the modeler about all aspects of the operation. It is the responsibility of the administrator to insure that the modeler understands the pending operation. Regardless of his skill level, the model belongs to the modeler. Most model management tasks are performed on the model management screen which is dedicated to the task.
The following operations must never be performed on a model while it is being used. If one of these operations is performed on a model while it is being used, there is a possibility of corrupting the CoreModel database.
System Administration For work to begin on a model, the model object must first be created by the administrator which is done by entering a new record in the models table. CoreModel will read the new record and create the model object. Although a model belongs to its modeler, only the administrator can create the model master which is done in the administration module. The model may be initialized at any abstraction level when created. The conceptual is the recommended level for a new model. The level at which a model is initialized must be established or agreed to by the modeler.
System Administration Because the destruction of a model is so easy, the system administrator is reminded that CoreModel is designed for professionals and does very little nagging. A model may be destroyed regardless of the amount of work that went into it simply by selecting the model from the list of models and pressing the button. Insure that nobody has the model open by displaying the list of current system users before destroying it. Also, if the Metadata Server has been activated, be sure that it is not cycling through an update. If anybody is using a model when it is destroyed, the database may be corrupted. Pressing that button will wipe all traces of the model from the system. When destroying a model, the system will allow the master to be retained while destroying only the model's internal objects. This allows a model to be cleaned out and re-used. To insure data integrity, when the system prepares to destroy a model object, it first closes any other screens that the administrator might have open.
System Administration Model replication is just as easy. Select the model from the list, type a new model name into the space provided, and press the button. Again, insure that nobody has the model open and that the Metadata Server is not currently rebuilding the web site or the dictionary so you do not corrupt your database. Every object in the existing model except change warnings will be replicated into the new model.
System Administration Because of the object oriented nature of the model, renaming a model is not simply a matter of changing the name on the master object. In addition to that, all of the model's internal objects must be relinked to the master object. Therefore, the rename operation requires that the model not be in use while it is being done. Renaming a model will cause CoreModel to reconstruct its data dictionary and web site. This presents no problem for any of the CoreModel subsystems, but the administrator should be aware of how the change may impact external systems.
System Administration A demonstration model is created as part of the system initialization. It may be deleted at any time and can be re-created as often as needed by pressing the button on the model management screen. When bringing a new modeler into the system the demonstration model might be an ideal way to allow him to experiment and learn. If there are multiple learners, a demonstration model may be created and then copied multiple times to new models with a new name for each modeler. The same method may be used to train the system administrator; i.e., numerous demonstration models with unique names might be created to allow experimentation with the Metadata Server and some of the utilities. The demonstration model is a model of the data presented by the CIA 2002 World Factbook. It is aimed at a generic database server. It is created at the logical level as though a modeler had been working on it for some time. However, it also contains work that has been done at the physical and implementation levels, so it may immediately be stepped to those levels. The demonstration is not intended to be free of errors. For example, it is intentionally over-normalized. Also, it is not intended to be complete. Some targets may require slight modifications to meet their peculiarities. However, it is generally valid enough conceptually and mechanically to be immediately instantiated.
_________________________ CoreModel manages all model namespaces. When the administrator creates a model, CoreModel automatically creates a protected namespace for each model. Namespace management is thus effected through the internal organization of CoreModel to provide complete freedom to the modeler and requires no administrative or operator intervention. The administrator may over-ride CoreModel's namespace management of data domains as explained in the data domain section of the documentation. This is the only allowable over-ride.
_________________________ Every user of the system is able to generate various kinds of output, and some operations performed by the user will generate activity reports. All of that output will go into the default output location unless the operator directs otherwise. For example, each time that a validation operation is performed on a model, the validation utility creates a validation report which it places in the default output location. If the default directory does not exist, the CoreModel will attempt to create it when needed. The startup default is c:\modeler, but the system administrator may change that in the system settings table. The system will recognize literals or unc's. The default output location is the same for every user so that the administrator can troubleshoot problems. If it specifies a local drive such as c:\modeler\, then each user will have a private output directory to prevent each from writing over the output of other people. However, a shared drive would have the advantage of centralizing everything. To specify the default output, refer to the output setting of the system settings section.
_________________________ This is a multi-user system. It is designed to permit multiple users and multiple unattended servers to connect to a single database and to simultaneously work in it. Install the system on each workstation and server and direct it to the installation database. If CoreModel's security has been turned on, the administrator can press the on-line button on the administrator screen to see who is currently using the system. It will display the user name, workstation, and when the logon was made. This utility is operational only if the administrator has created personnel objects for everybody who uses the system. If system maintenance must be done that can be done only when nobody is using the system, the system may be locked from the configuration screen. See the system settings section for instructions.
_________________________ CoreModel has a security sub-system which may be toggled on or off at will by the administrator. Security controls access to the CoreModel installation and it controls movement and operations within the system after access is granted. CAUTION: System security must not be turned on until the administrator is familiar with the documentation and valid backups have been made of the installation database! Operation of the security sub-system is not required by CoreModel and its use is entirely at the discretion of the administrator. The modeling operations are entirely independent of it and unaffected when it is not used. A single user installation, in which the administrator/modeler does not have time to manage security, can easily do without it. Also, very large organizations which have custom security for enterprise-wide control can use only their security as a front end for CoreModel and leave CoreModel's turned off if they wish. Some management information is obtained from the operations of the security sub-system. For example, if security is not maintained, pressing the button that lists the people who are currently using the system, will show nobody. In organizations that use their own front end for CoreModel, it might be a good idea to enable the domain type security (see below), which is very unobtrusive, to allow CoreModel to know what is going on. For information on changing the security settings, including the security toggle, refer to the system settings section.
System Administration CoreModel has a simple security toggle which the administrator can set to on or off just by pressing a button in the configuration screen. Instructions for the operation of the security toggle are found in the system settings section. When CoreModel is installed, the initialization procedures turn the security sub-system off. When the administrator turns it on to it awaken it, it immediately applies controls to the installation and begins watching activities. If there are people working in the installation when security is turned on, their movements may be impeded if security preparations are incomplete, so if security will be turned on in a multi-user installation, it should be done immediately and before modelers begin using the system. Caution! Before turning on the security sub-system, it is extremely important for the system administrator to understand how it controls access to and movement through the system. When the security is turned on, the system will be protected from everybody, including whomever turned it on. Therefore, before turning the security on, the administrator should make a backup of the database and then make himself known to the system by entering the necessary information as outlined in the following passages.
System Administration When the security toggle control is changed, the administrator will notice another control on the same screen which controls the security type. The security type specifies the type of foundation control that is used by the security sub-system, and it may be altered at will by the administrator. There are three types of security; domain, login, and a combination of the two. The organizational infrastructure must support the domain access control before it can be used in CoreModel. If the organization is using domain controlled access ( which is used in most large organizations ), then CoreModel can tie into it. The domain type is the easiest to use if the infrastructure supports it. When this type is used, because it is silent and very unobtrusive, CoreModel appears to have no security when he is loaded by somebody whom he knows, but when somebody whom he does not know tries to load him, they are suddenly denied access. He does that bit of magic simply by talking to the domain. When somebody loads him, he asks the domain who the person is, and searches his files for a matching personnel object. If he finds a personnel object with characteristics matching those of the person, and which is authorized access to the system, he lets the person have access. The login type of security assumes that the organizational infrastructure is insecure and uses only CoreModel's internal system. When somebody loads CoreModel, they see a login prompt, which they would not see under the domain type, in which they must enter security information. The system then looks for a personnel object that matches the characteristics of the person logging in. If one is found, and it has entry authorization, entry is granted. The login type may also be used in organizations where the infrastructure is simply not available to support domain security. A combination of the two types may be selected which increases the level of security greatly ( along with the headaches for the system administrator ). Under this type, the person is presented with a login screen and required to log in. The system will look for an internal personnel object which matches the person logging in. If one is found and it is authorized for system access, CoreModel will then ask the domain controller if it has granted access to that person. If so, the person is finally allowed to use the system.
System Administration The security subsystem includes personnel objects which must be created by the system administrator. A personnel object is required for each person who will use the system. (The system administrator must insure that his personnel object has update privileges in the administration module. ) The login process looks for a personnel object with characteristics that match whomever is logging in. After access is granted, the various clearances within the object determine what the person can do within the system. ( For information on maintaining these objects, refer to the personnel objects topic of the system tables section. ) One of the characteristics of the personnel object is the password. It is required and a blank password cannot be used. In addition to its characteristics, the personnel object also carries the module clearances. A person must have clearance specific to a module before he can load that module. A personnel object gives a person access to the system, but he can move into only those modules to which the personnel object has been granted access. The module access clearance is affected by clicking on the objects in the displayed list. ( Caution. The system administrator must have update access to the administration module. ) Additionally, the personnel object carries the type of module access. The access types are update, read only, or none. The clearance must be granted either for update privileges or for read-only privileges. Read-only privileges will not allow updates within a module. A modeler's personnel object must be given update access for each of the modeling modules if he is to work in the modules. Read only access cannot be granted for the admin module. (An advisable administrative technique is for the administrator to give himself read-only access to the modeling modules to avoid accidental changes to models when he is assisting a modeler, but he must have update access to the administrative module.)
System Administration The final layer of the security system controls access to each model after the other layers are passed. The model access table is a series of links between personnel objects and model objects. A modeler can open a model only if there is a link between his personnel object and the model object in the model access table. He can be allowed to open any number of models or none. The personnel records and model objects must be created first because the system uses that data to fill the drop down selections on the model access screens. The model access table must be prepared when the security system is turned on or modelers will be locked out of their models. For instructions on maintaining the table, refer to the model access table in the system tables section.
System Administration The failsafe feature of the security system is a high security feature which has been deactivated in all systems because it intentionally puts the database at risk. Its danger is unsuitable for normal operations.
System Administration The system may be locked for maintenance through the setting in the configuration screen. Although system locking is not a security feature, it is covered in this section because it must be handled by the security sub-system. The lock will remain on until it is turned off. The system lock is operational only when the security system is turned on. While the system is locked, only the system administrators can open it. Anybody else attempting to log in will be told that the administrator has locked the system temporarily for maintenance. If others are working in the system when it is locked, the system will allow them to continue working until they exit the system. In a large installation, after the lock is applied, the on-line utility should be used to see if anybody was inside the system when the lock was applied. Also, although the lock becomes effective immediately, infrastructure latency should be considered in a large installation.
System Administration CoreModel security can be overwhelming for a new administrator. It is designed to be able to serve as a high quality multi-level enterprise level security system that can manage the complex activities of a large installation serving multiple departments that are geographically distributed around the world. Therefore, before turning it on, one should consider whether or not it is needed. For a single user operation, where the user does not have time to become a qualified system administrator, but security is needed, the following steps are suggested as a general guide. The administrator should:
_________________________ There are a number of system setting values (parameters) which CoreModel reads during startup to determine system characteristics. The values are established during CoreModel's initial installation, but because they set important characteristics of the system, the administrator should review all of them as soon as possible. They control characteristics of the entire CoreModel installation on all workstations and servers. The system settings that are covered in this section are in addition to the system tables, some of which are also used by the system. To find the system settings, load the administration module and press the config button. The configuration screen will be displayed. On it, highlight the name of a control group and that group of controls will be displayed. ( The Metadata Server has a separate set of settings which are maintained from the server's own screen in the administration module. See the Metadata Server section for its configuration. ) When the administrator modifies and saves a setting, the setting immediately becomes available to using systems. A modified setting must be loaded into memory by the using system when the using system loads. Therefore, if a setting is changed while the system that needs it is loaded, that system should be unloaded and reloaded.
System Administration The daily message toggle can turn the feature on or off. If turned on, a randomly selected message will be displayed each time that the system is loaded.
System Administration In a client/server environment, it is possible for a workstation or server to be overwhelmed by accidentally pulling back an inordinately massive data set. To guard against that kind of catastrophic system crash, CoreModel contains a maximum data set size setting. The system is initialized during installation with a maximum data set size of twenty thousand records per data set. It is extremely unlikely that most modeling operations will require data sets larger than the default. However, the ability is provided for the administrator to change that value in the system configuration screen. Changes to the maximum data set size should be done cautiously. There can be no warning of a system crash that is caused by getting too much data. The maximum value is the value for each data set, so it is possible for an operator or server to retrieve many times that number of records when multiple data sets are being used. Setting the value too low may prevent an operator or server from retrieving the records needed to do the job at hand. The optimum value for the setting is best determined by the experienced judgement of the system operator after some experimentation. The default value of twenty thousand has been found to be generally effective.
System Administration Database connection parameters are maintained in an ini file on each computer. When CoreModel is loaded on a computer, it looks for a local ini file and reads the values from it to find its database. Those current database connection parameters are displayed on the setting configuration screen. Every time that the system is closed, its current database settings are saved into the local ini file so that the next startup will be with the last set of changes. All workstations in a CoreModel installation may be connected to a different database by changing the settings in the ini file. ( The CoreModel administrator is assumed to be certified, so he understands database connections.) Before making any changes to the ini file, the old file should be copied to a new name in case a restore is needed. Since the database connection settings are in an ini file, a text editor may also be used to change the file. When CoreModel loads, if the ini file is missing or corrupted, he will initialize a new one. The system administrator should maintain a backup to restore ini files that become damaged or deleted. If CoreModel halts a load and requests directions to his database, the administrator should shut the system down, repair the ini file, and then restart. For general observations on the CoreModel database, see the database backend section. Fail Safe The failsafe feature is part of the security sub-system and it is deactivated in all systems until further notice. The failsafe settings will not be covered in the documentation.
System Administration The fonts in all data browser screens and in all info screens for the entire CoreModel installation are controlled from the configuration screen. ( Each screen also has a temporary over-ride. ) There are settings for the font size, face, and bold.
System Administration If uncontrolled, the number of records accumulating in the log would eventually overwhelm your system, but the system cannot be allowed to purge records because of security requirements. The log size setting specifies the maximum number of records allowed. When the system administrator presses the truncate button, the log is reduced to the size specified by the setting. When determining the maximum log size, the administrator should consider the importance of the log in the management of system security and in debugging problems with the system. In a large installation running on a database server, million record logs are not uncommon. If anything goes wrong, or if supporting documentation of an issue is needed, the log can be invaluable. For more information on the log, see the log section.
System Administration See verbose logging in the log section. If verbose logging is toggled to false, a minimal amount of activity is logged such as somebody loading and unloading the system. If it is toggled to true, which is the default, detailed activity records are kept for debugging and security issues. Verbose logging is recommended. ( Due to their potential for a high rate of around-the-clock log activity, the servers have their own verbose logging controls. ) For more information on maintaining the log, see the log section.
System Administration
System Administration The name and address of the organization may be entered, but are not required. If they are entered, CoreModel will use them to replace the CoreModel header on web pages. Each line is free form text and each may be up to one hundred characters long.
System Administration The output default is the path to the disk location where system output will be placed by default. All system output will go into there unless the operator specifies otherwise. For additional information, see the default location section.
System Administration The security sub-system may be toggled on and off. If others are using the system when the security toggle is changed, have them exit and reload. Read the security section.
System Administration This sets the type of foundation security that the system will use. Read the security section.
System Administration To lock the entire installation, toggle this setting to on, or to off- to unlock it. The lock will remain on until it is turned off by the administrator. System locking is handled by the security sub-system, so refer to that section for details. The system lock toggle is located with the security settings on the configuration screen.
_________________________ The system administrator must maintain various data sets and objects which are used by all the modelers and by CoreModel itself. They are listed on the administration module screen beside three buttons labeled create, edit, and browse. If one of those buttons is pushed when one of the object names is highlighted, the create, edit, or browse screen will be displayed for the highlighted object. The three types of data screens that are used by the modelers are used by the administrator; the create, edit, and browse screens. Refer to the data screens in the modeling section for descriptions and operations. Each screen may be loaded by first highlighting an object and then pressing one of the labeled buttons. For example, highlighting the connection selection, and then pressing the create button will open the create screen for the connection objects. Multiple objects may be loaded simultaneously. For example, if all objects are highlighted and the create button is pressed, the create screens will be opened for all of the objects. The browse screen is particularly worthy of the administrator's attention. Getting started in a system as large as CoreModel can be an overwhelming task. To help the administrator get started, CoreModel writes hundreds of records into the administrator's data tables during installation and initialization . A few serve only as examples, but most, such as the data types, can be left in place and expanded upon.
System Administration Seen by all personnel when turned on. The messages that ship with CoreModel are primarily aimed at developers and are of the humorous type. They may be replaced.
System Administration Used system-wide by the modelers. Domains are objects which can be created within each model for use within the respective models. Additionally, standard domain objects may be created by the system administrator which may be used throughout the system in all models. The administrator can acquire ownership of an existing domain that was created by a modeler by displaying and saving its record. It will then be available for use in all models system-wide, but is thereafter controlled by the administrator.
System Administration Used system-wide by the modelers and by CoreModel. Be sure to cover any types that are peculiar to local target systems. Duplicate data type names may be entered as long as each is not duplicated within a DBMS. The server names must precisely match the names of the server objects so the modelers can test their models. A recommended technique is to create the server object first, copy its name, and paste the name into each of the data types.
System Administration Used system-wide by the modelers. Database connection objects are used by the modelers to connect to a database for various tasks such as model instantiation. The subject of connecting to data sources is addressed in the CoreReader documentation which can be seen on the CoreReader website http://CoreReader.com/ . The same documentation also has sections on numerous specific data sources. It is important to know that each type of connection has its own characteristics and each returns different data from a given data source. Also, each data source behaves differently with the different types of connections. The administrator should be aware of these characteristics to assist the modeler with understanding such things as model imports. There is a test button on the connection screen to allow the administrator to test a connection before it is saved. If a connection cannot be established, or if the connection is established, but the required operations fail, then the first thing to check is the security in the other system to insure that appropriate access has been granted. Data connections are temporary. CoreModel makes no permanent connections to other systems, and makes connections only when told to do so. To maintain the security of other systems, the modeler is allowed to see only the connection name when establishing his data connections, so short descriptive names are recommended for the connection objects to assist the modeler in identifying the one needed on his screen. There are four types of data connections: ODBC, OLEDB, DML, and CDO. The type of connection and the brand of the data source will determine the nature and kind of information that can be exchanged, as explained on the CoreReader web site. The modeler should be made aware of the type of data connection that he will be using, and he may need instruction on the characteristics of that type of connection. A DML connection type is, of course, simply the name and location of a text file. However, the connection must be set up so the modeler can use it to import into a model or to instantiate a model. OLEDB connections require the extremely heavy Microsoft OLEDB data components which are not distributed with CoreModel. If an OLEDB connection is attempted, and the OLEDB software is not present in the computer, CoreModel will display a notification message to that effect. OLEDB may be obtained by the CoreModel system administrator from the Microsoft web site as part of the MDAC download. Also, it may be obtained by installing CoreReader on the computer. (CoreReader is a recommended tool for all data modelers and administrators.) ODBC connections are the industry standard database connections. When connecting to AxleBase, the CDO connection type should be selected. No connection software is required for AxleBase because it is an embedded database manager. When connecting to an AxleBase database, enter nothing in the path. In the server box, enter the path to the master database. The name of an AxleBase database is also the name of the directory in which it is located. Do not include database names in the path. For additional information about AxleBase, see the AxleBase website at http://www.AxleBase.com/
System Administration These are used system-wide by the modelers and by CoreModel. Database managers are software systems that manage databases. They are sometimes referred to as database servers, RDBMS's, or sql servers (not to be confused with the Ms. Sql Server which is only one of the many available sql servers). The brand names must precisely match the names that are entered on the data types so the modelers can test their models. A recommended technique is to create the server object first, copy its name, and paste the name into each of the data types.
System Administration These are the types of databases that might be targeted. Not only might the operators use the database types, CoreModel uses the database types for such things as model validation. Edit-ability is presented to the system administrator to allow him the management flexibility that he might need, but if the initial database types are changed or deleted, then some of CoreModel support features will cease operation.
System Administration Used system-wide by the modelers. See the help sub-system section of the documentation.
System Administration The model access table is part of the security sub-system. Please refer to that section for an explanation. There are only two fields on the create screen. A login id and a model name. Each is a drop down list and both must contain values before saving the record. Creating a record consists of selecting the two names and saving the record. Since the record is so simple and contains only the two pieces of data which are simply selected from lists, an edit screen would be pointless; records are changed by deleting and creating a new record.
System Administration Used system-wide by the modelers and by CoreModel. The model levels correspond to the abstraction levels. They are used by CoreModel for various purposes including determining the module in which to open a model. Since the concepts are fixed in the construct, this table cannot be edited.
System Administration The codes in this table specify the current working state of the model and are intended to serve as management flags. Their use is entirely at the discretion of the administrator.
System Administration This table is not currently used and cannot be edited. All of the models in the system will be of the relational (r) type.
System Administration See the model management section.
System Administration Used system-wide by all modelers and by CoreModel. These are the naming standards that are applied by the modeler by running his standardization utility. The available standards are fixed but the values are maintained by the system administrator. Each standard record displays a list of permissible values to guide the administrator. The available name standards are:
System Administration Used system-wide by all modelers and by CoreModel. These are the types of objects that are in the system. The object type codes are used in the objects and in the object management information. It is unlikely that the object type codes will need to be updated after system installation, but edit-ability is presented to the system administrator to allow him the management flexibility that he might need. It is recommended that the initial object types remain unchanged because not all of the type codes can be changed on all of the objects in the system, which means that changing a master code might break some links.
System Administration After security is turned on, a person can access this system only if it contains a personnel object with his attributes. The administrator controls system access through these objects. Therefore, before turning the security on, the administrator should first insure that the required personnel objects have been created. The login ID, first name, last name, and password are required in each object. A blank password will not be accepted. Each login Id must be unique. At the bottom of the screen is a list of the system modules. Clicking on the number tabs displays additional modules. A check must be placed in one of the boxes next to each module that a person must be able to access. A check in the update box allows update ability or a check in the read box restricts the person to just viewing data in the system. If the default check is left in the none box, then access to that module will be denied. When managing security, an accepted rule of thumb is to give as little as possible. Also, it is a lot easier to give than to later take away. (Caution. The administrator must have access to the entire system before closing the system. ) For general security information see the security section .
System Administration Used system-wide by all modelers and by CoreModel. These are the types of processes. (See the process notes in the modeling section of the documentation.) Process types are especially important to CoreModel. Edit-ability is presented to the system administrator to allow him the management flexibility that he might need, but if the initial process types are changed or deleted, then CoreModel will not be able to properly use them and some system functions will cease operation.
System Administration Used system-wide by the modelers. As part of its initialization process, the system created the basic relation type records. It is recommended that those system generated relation types not be changed or deleted because they are minimal and basic to the models. Deleting or changing them is left to the system administrator's judgement and will not cause errors, but they are used as initialized by the system in support of the modeling activity. Additional relation types may be created. A new relation type will have no impact on the system.
System Administration Used system-wide by the modelers. Data source types are not currently used by CoreModel, so the initialized table may be changed as the administrator sees fit.
System Administration Used system-wide by the modelers and by CoreModel. See the system settings section.
_________________________
System Administration Logging is always operational. The log cannot be edited, and can be viewed only by the system administrator.
System Administration As part of the security design, the system will not automatically truncate its log, so this is a matter that requires the attention of the system administrator. To assist in this matter, CoreModel checks the log size during each system load. If the size exceeds the truncation point by more than fifty percent, an alert message is displayed. The log is truncated by pressing the button on the administration form. Truncation of the log is based upon the maximum log size which was previously set in the config screen. Truncation will reduce the number of records in the log to approximately one quarter of the max size. Those will be the most recent records. For guidance in setting the maximum log size, refer to the log size setting in the system settings section.
System Administration Verbose logging is turned on when the system is installed and may be turned off in the settings form for minimal activity logging. If verbose logging is toggled to false, a minimal amount of activity is logged such as somebody loading and unloading the system. If it is toggled to true, which is the default, detailed activity records are kept for debugging and security issues. In a heavily used multi-user system, detailed logging can quickly create a large log. However, consider how important the log is in the management of your system security and in debugging problems with the system. If anything goes wrong, the first place that you will look is in the log. Perhaps the best way to make a decision about verbose logging would be for the administrator to turn verbose logging on and watch the log for a day or two. If the log activity seems beyond the ability of the equipment and database, then it might be best to leave verbose logging off until it is needed. If the capability of the infrastructure seems more than adequate, then perhaps verbose logging might be left on. The system administrator should include anomalous activities in his decision. For example, a single import could write thousands of records into the log within seconds. If it appears that the infrastructure cannot handle verbose logging, and it must be left off, it can always be turned on later. If unexplained events occur, verbose logging can be turned on until their source is traced or it can be turned on for specific system tests. For guidance in setting the verbose logging toggle, refer to the verbose logging setting in the system settings section.
_________________________ CoreModel contains in-context help, prompts, warnings, and the documentation. (CoreModel intentionally does not use the microsoft help system.) However, the system administrator may find all of that either inadequate or not compliant with the needs of the local organization. Therefore, CoreModel contains a help sub-system which allows the system administrator to enhance the existing on-line help. In the system administration module there are a help create screen and a help edit screen which allow the administrator to save help messages in the system. On those screens are drop-down boxes which contain the categories of help message. For example, there is a columns category. Selecting the columns category, entering a message, and saving it will cause that message to be displayed when the info button is pressed on any screen pertaining to model columns. The message will appear along with any system messages that are presented for pertinent screens and data tables. A relational database should not contain apostrophes (the current fad is to refer to them as single quotes), so if an apostrophe is entered in a help message, CoreModel will double it before saving it. Only one message can be saved in each category, but a message can include carriage returns to make it appear to be multiple messages. Help message categories are determined by the system. Each message can be of an indefinite length in addition to any existing system information.
_________________________ A CoreModel installation includes an active interface which makes CoreModel capable of receiving and acting upon administrative commands and configuration parameters from anyplace in the world via any type of communication or network, including the internet.
System Administration Before using this feature, the system administrator should be confident of all facets of the security issues. CoreModel cannot secure any part of the communication links. CoreModel's primary security responsibilities are covered by the internal remote control server toggle. If the remote control server is not turned on, the system cannot be controlled remotely. Turning it on, therefore, deserves some thought and preparation. The remote control configuration includes a remote id and a remote password. These must be set and they will be used only one time. The remote control XML schema and sub-system require an id and a password in every transmission to the CoreModel installation. When parameters are received by the remote control sub-system, if the transmitted id and password cannot be verified against the saved id and password, the processing is stopped with a logged error message. The system will not accept a blank id or a blank password in the transmission. The system requires that a new remote id and a new password be received in every transaction. The new id and the new password will be the required id and password on the next server cycle, and they will be used only one time while receiving a new id and a new password.
System Administration The remote control of CoreModel is accomplished via a disk interface. A location is designated as the inbox for CoreModel and the internal remote control sub-system will watch that location for input. The input may be delivered by any means that the administrator chooses. However, CoreModel's Web Site Administrator provides a built-in method. When it creates a web site, if the system administrator allows it, it will create a remote control page on the site. That control page can then be used to transmit commands and parameters to CoreModel. Control communication is affected through schema based XML technology. The remote control server expects to receive only its own schema and will reject anything else. If only CoreModel tools are used for remote control:
System Administration To use the remote control feature, it must first be configured and turned on. To do this:
Controls
The remote configuration controls are so similar to those of both of the Metadata Servers that much of the instructions for those servers will suffice for the remote server. Note that this server also requires a remote id and password which are discussed above under security . It will be noted that there is not a separate logging toggle for this server as there is for the Metadata Servers. This is by design. Caution! The system administrator must be aware of the location that is given to the server. There should be no other files in that location and it must not be used by other systems. The CoreModel server will assume ownership of the location and anything that the server does not recognize will probably be destroyed. Pointing any CoreModel server at a computer's root directory on the primary disk would be especially unfortunate.
_________________________
System Administration The metadata server is two robotic processes which are designed to run without human assistance on their own computers. They watch the database as modelers work and automatically maintain the metadata in a data dictionary. Metadata generation is accomplished internally by CoreModel through model object translations, so the modelers and operators do not need to be concerned about it. The metadata server interfaces to the rest of the CoreModel system to receive metadata as it is generated without human intervention. The server then presents the metadata to the external world through self documented interfaces. The metadata interface is actually maintained by two separate servers. Although they can be run locally and manually, these servers are designed to run autonomously on unattended computers to maintain the data repository for an organization. One server is a robotic web site administrator which creates and maintains a complex CoreModel web site. The other server maintains a dictionary interface which is more appropriate as an interface to external computer systems which require metadata. Each server may be run on a separate computer, or they may be run on the same computer. (Note! Do NOT instantiate multiple web administrators. Do NOT instantiate multiple dictionary servers. Although difficult to do, it is possible to instantiate and run multiple metadata servers, but they would conflict and corrupt the publications. ) Since the CoreModel system automatically and autonomously administers its web site and data repository, the metadata is dynamically driven by changes as they are made by data modelers. The server modules within CoreModel are toggled on or off and have parameters that are controlled by the human system administrator, but the CoreModel servers otherwise independently maintain the web site and the data dictionary so that a link is created and maintained between the metadata and the models. The metadata server provides unidirectional links from the models to the external world. When any CoreModel server is activated, it reloads its parameters on each cycle. This allows the system administrator to control the server characteristics and operations from a remote console and it also allows him to change features without killing a busy installation's metadata server. CoreModel checks for a call to the metadata server only when he loads. Therefore, to start a server, configure it for operation on the workstation, which can actually be done from a different computer, and then start CoreModel on that workstation. When CoreModel loads on that workstation, he will see that he is the designated server, and will take over. When a server is brought on line, it will stay alive until it is brought down. To kill a server, turn it off in the configuration settings and then unload CoreModel on that computer. When CoreModel is reloaded, the server will not start.
System Administration The release of a model for publication is a multi-level operation. It must first be released by the modeler, then it must be released by the system administrator, and finally the system administrator must tell the server to publish the models. The modeler gives approval by pressing the publish button and the administrator does so by editing the model object. Additionally, the administrator must configure the server to publish any models that have been released. The dictionary presents an interface which is expected to be used by external systems and agents, with perhaps dynamic links being formed by external systems. Therefore, publication cannot be rescinded by the modeler. However, the model is dynamically linked to the metadata interface, so changes made within the published model are transmitted to the interface to allow updates to the published dictionary. This behavior may be over-ridden by the administrator for emergencies. Although he cannot alter the modeler's publication, he can alter his own, so that the publication is rescinded. Caution: The administrator must be aware of the impact of such an action on systems that may have linked to CoreModel through the dictionary interface.
_________________________ When properly configured the CoreModel system will create and maintain its own web site on the internet. Of course, a web server must be available for support. See the previous Metadata Server section for a description of the web administrator.
System Administration
Creating or updating a large web site can take some time, and the process should not be interrupted. Although unlikely, interruption of the process could corrupt the CoreModel database, and most certainly will corrupt the web site. In the development instance, with fourteen thousand elements creating several hundred web pages, each update required several minutes. Depending upon the size of your operation, the CoreModel web administrator could create hundreds or thousands of web pages. In most cases, if an error is encountered, all of the CoreModel servers have been told to log the error and shut down. The CoreModel instance will continue running, but the server that was running under that instance will have been shut down. The exception is a "not found" error. If the server is working on a site, and suddenly an object cannot be found in the database that was assumed to be there, CoreModel assumes that somebody or something deleted it, logs his irritation as an error message, and continues working. After CoreModel creates its web site, the local web server may be started and directed to the CoreModel web site and the CoreModel home page. Thereafter, the web server and the CoreModel web site administrator will maintain a current version of CoreModel data on the net.
System Administration The configuration controls for the web site administrator sub-system are located in the configuration screen which is loaded from the administration form. The administrator must configure and activate the process which then allows the CoreModel web site administrator module to begin functioning. The web site administrator is actually an automated unattended autonomous server. (The web site administrator is separate from the export utility that allows operators to export data from the system into web pages.)
System Administration These are the server controls which are located on the configuration screen:
System Administration The logging of the web administrator is controlled independently of the rest of the system. A high speed server can quickly create a tremendous log file of unwanted proportions. However, the system administrator may require logging of those operations when the rest of the system is not being logged, so the web administrator has its own logging control. A factor to consider when deciding whether or not logging should be on is that an unattended server writes error messages into the log, so if the logging is turned off, the error messages will be lost. As noted elsewhere, the Ms. Access backend which ships with CoreModel is not suitable for heavy usage. If the installation is being used by only one person, it might be a good idea to leave the web administrator logging turned off to insure that Ms. Access is not overwhelmed by CoreModel's activity reports.
System Administration The system administrator can specify the location of the web site. Be sure that CoreModel has security access to allow it to create the location. It will create the last element of a path if necessary, but will not create an entire path. The system will publish all data models, supporting data, administrative data, and documentation on the site. System security items are not published. In a heavily used system with many models, the site may become very complex. Manual intervention in the site is pointless because CoreModel may erase your work in its next cycle. Caution! Be aware of the location that you specify. Also, insure that there are no other files in that location and that it is used by no other systems. CoreModel will assume ownership of the location and, since he is a tidy housekeeper, anything that the server does not recognize will probably be destroyed. Pointing the server at a computer's root directory would be especially unfortunate.
System Administration The web administrator can run on a machine on which modeling is done or it can run on the system administrator's computer, but this is not recommended because the administrator may conflict with the operator's needs. Therefore, CoreModel is architected to allow the system administrator to set up an unattended web controller computer for the CoreModel system. This allows the computing load of maintaining the web site to be offloaded to a computer other than the administrator's machine, and allows the web administrator module to be run around the clock as a server. Set up the user login that will be used on the server machine, and enter that login name as the controller on the web configuration screen. When CoreModel is loaded, if it sees that the web site administrator is the current login, it will start the web administrator module running on that computer. A CoreModel instance can then be left running continuously on the unattended machine as a web controller to keep the web site updated. Only the CoreModel instance that is specified as the web controller will maintain the system web site, so that other users can log into the CoreModel system without the web administrator prompts.
System Administration There are three toggles that control what is put on the site. The admin toggle will allow publication of the administrative data. (Defaults to yes.) The control toggle allows CoreModel controls on the site. (Defaults to yes but not recommended.) The models toggle will put the models on the site. (Defaults to yes.) Model publication on the web site is controlled entirely by the administrator.
System Administration There is a toggle in the controls that allows you to specify that the web process will be running on an unattended computer. If that toggle is set to yes, which tells the system that it is running on an unattended machine, CoreModel will not pop up message boxes when the web process runs. If a message box were to pop up on an unattended machine, the server would be locked up until somebody happened to notice the need for a response. However, if you will be working in the same system, you might like to set it to no so that it will let you know when it is working. You certainly do not want to shut the system down in the middle of a site update because that might corrupt your files. Also, you might like to know when the system will be contending for your computer's resources, so you would want to tell it that it is running on an attended machine so it will let you know when it cycles. Note however, that if the process is running on an unattended computer, you MUST let CoreModel know. If you don't let it know that it's on an unattended machine, the server WILL lock up while waiting for a person to respond to its messages.
System Administration An update frequency must be selected. The web administrator will update the entire web site at the beginning of each period selected. This insures that the web site reflects the current state of an active system by automatically and continually updating the site. The cycle frequency is actually a period of time between the end of the last cycle and the beginning of the next. The system is designed in this manner to insure that the load on the database server cannot over-ride CoreModel's control. For example, a heavily loaded infrastructure might force the CoreModel server to slow to the point where it takes ten minutes to update the site, but if a cycle interval of five minutes is in use, when the ten minute update is complete, CoreModel will wait the designated five minutes before beginning the next update instead of being forced into a continuous cycle by the slow infrastructure. When shutting down an installation for maintenance, the server cycle must be noted and accounted for. When a server is told to shut down, it will shut down after completing its next cycle. In a large distributed installation, the administrator can use the log to watch the activities of multiple servers to know where they are in their cycles. For long cycle times, it may be safe to unload the server during its rest period if the operation is sufficiently far ahead of its next cycle.
System Administration The script parameter specifies what type of scripting will be used by the target web server on the web site. At this time, the objective is to support only asp and cgi due to the limited number of manhours available to the project.
System Administration See the content control
System Administration The default home page for the CoreModel site is index, but this may be changed to whatever you want. Do not include a path or a suffix.
System Administration The CoreModel web site may be linked into a master web site by specifying a return page that is different from the CoreModel home page. If the CoreModel site is to be autonomous, make the return page the same as the home page. If the CoreModel site will be linked into a larger site, then enter the entire path or url for the page to which you want users to return from the CoreModel home page. Remember: if the CoreModel site is autonomous, the return page must be identical to the home page which is simply a name, but to link the CoreModel site into a larger site, the entire path or url must be entered as the return page.
System Administration Active Server Pages are a Microsoft technology which may be used on a web site. This is absolutely not an endorsement of the Ms. technology, but is included only in case somebody needs to use it. Do not attempt to employ it in a CoreModel installation unless you already understand it. The active server page technology within the web technologies is not easy to master, and when presented within the complexity of a CoreModel installation, it might be overwhelming. As was covered in a previous section, CoreModel contains a remote control server which is linked to the web through schema based XML. Therefore, a mechanism is required to dynamically maintain updated XML to drive the remote control server. The web administrator controls include a script selector. If the ASP script is selected in a properly configured system, the web administrator will produce an ASP page instead of a CGI page for the remote controls. Additionally, it will create an ASP page to feed the control page parameters into an engine to produce the control schema. A schema engine can be constructed locally by the system administrator. It simply needs to be able to be run on demand by the ASP page, accept the parameters from the ASP page, and create a schema that conforms to the CoreModel standard in any location specified by the ASP document. However, since I had to create a schema engine for the testing, it is made available under the CoreModel license and it may be obtained by request. The provided engine is a standard Ms. Windows DLL.
System Administration The test button will run the server through a single operations cycle. This will test the system and the entered parameters, and will create a complete web site based upon the parameters and the data currently in the system. When the test is run, the system may change parameters on the screen, but no changed parameters will be saved unless you press the save button. For example, you may see the activation toggle change from off to on, but the changed parameter will disappear when you close the screen unless you save it. To protect production installations, the system will not allow a test on a computer where the server is already loaded and cycling. If the system detects a loaded server when you press the test button, it will tell you and abort the operation. To run the test, you must either go to a different computer or you must turn the server off and reload CoreModel.
_________________________ The CoreModel system contains a Repository Server which is model driven and which may be toggled on and off by the system administrator. It will present the metadata repository in a public interface which is more suitable for external links by or from metadata consumers, whereas the web site administrator presents an interface more amenable to human consumption. It is an unattended and autonomous system which will begin automatic processing when placed in service. The configuration controls for the server sub-system are located in the server screen which is loaded from the administration form. The administrator must configure and activate the process which then allows the server to begin functioning. The repository server serves only as a metadata dictionary. It does not publish ancillary items of interest as does the web administrator server such as data types, log files, et etc.
System Administration When properly configured the CoreModel system will create and maintain a data dictionary. The operation is similar to that of the web administrator server. Operation:
Creating or updating the data dictionary is slightly faster than the web operations, but it too should not be interrupted to avoid corrupting the database. In most cases, if an error is encountered, all of the CoreModel servers have been told to log the error and shut down. The CoreModel instance will continue running, but the server that was running under that instance will have been shut down. The exception is a "not found" error. If the server is working on a site, and suddenly an object cannot be found in the database that was assumed to be there, CoreModel assumes that somebody or something deleted it, logs his irritation as an error message, and continues working.
System Administration The configuration controls for the server are located in the configuration screen along with the web administrator controls. The administrator must configure and activate the process by entering and saving the required data which will allow the server to begin functioning.
System Administration These are the server controls which are located on the configuration screen:
System Administration The model control behaves differently from the web model control. If it is set to yes, it will over-ride all of the model publication settings, and the server will publish all models in the dictionary regardless of individual approvals.
System Administration The system administrator must specify the location of the dictionary as a valid path. Be sure that CoreModel has security access to allow it to create the location. It will create the last element of a path if necessary, but will not create an entire path. The system will publish all data models. Manual intervention in the dictionary is not needed and is useless because CoreModel may erase your work in its next cycle. Caution! Be aware of the location that you specify. Also, insure that there are no other files in that location and that it is used by no other systems. CoreModel will assume ownership of the location and anything that the server does not recognize will probably be destroyed. Pointing the server at a computer's route directory would be especially unfortunate.
System Administration The repository server can run on a machine on which modeling is done or it can run on the system administrator's computer, but this is not recommended because the server may conflict with the operator's needs. Therefore, CoreModel is architected to allow the system administrator to set up an unattended data repository server computer for the CoreModel installation. This allows the computing load of maintaining a data dictionary to be offloaded to a computer other than the administrator's machine, and allows the server to run around the clock. Set up the user login that will be used on the server machine, and enter that login name as the controller on the configuration screen. When CoreModel is loaded, if it sees that the server logon id is the current login, it will start the data repository server module running on that computer. A CoreModel instance can then be left running continuously on the unattended machine as a server to keep the data repository updated. Only the CoreModel instance that is specified as the server will maintain the dictionary, so that other users can log into the CoreModel system without the server prompts.
System Administration There is a toggle in the controls that allows you to specify that the process will be running on an unattended computer. If that toggle is set to yes, which tells the system that it is running on an unattended machine, CoreModel will not pop up message boxes when the process runs. If a message box were to pop up on an unattended machine, the server would be locked up until somebody happened to notice the need for a response. However, if you will be working in the same system, you might like to set it to no so that it will let you know when it is working. You certainly do not want to shut the system down in the middle of a site update because that might corrupt your files. Also, you might like to know when the system will be contending for your computer's resources, so you would want to tell it that it is running on an attended machine so it will let you know when it cycles. Note however, that if the process is running on an unattended computer, you MUST let CoreModel know. If you don't let it know that it's on an unattended machine, the server WILL lock up while waiting for a person to respond to its messages.
System Administration An update frequency must be selected. The system will update the entire dictionary at the beginning of each period selected. This insures that the dictionary continually reflects the current state of an active system. The cycle frequency is actually a period of time between the end of the last cycle and the beginning of the next. The system is designed in this manner to insure that the load on the database server cannot over-ride CoreModel's control. For example, a heavily loaded infrastructure might force the CoreModel server to slow to the point where it takes ten minutes to update the site, but if a cycle interval of five minutes is in use, when the ten minute update is complete, CoreModel will wait the designated five minutes before beginning the next update instead of being forced into a continuous cycle by the slow infrastructure. When shutting down an installation for maintenance, the server cycle must be noted and accounted for. When a server is told to shut down, it will shut down after completing its next cycle. In a large distributed installation, the administrator can use the log to watch the activities of multiple servers to know where they are in their cycles. For long cycle times, it may be safe to unload the server during its rest period if the operation is sufficiently far ahead of its next cycle.
System Administration The self describing nature of the interface which is constructed by CoreModel provides a way for external systems to find their way around within the data dictionary with minimal human intervention. The entry point is manifested as the XML schema in meta.xsd. Therefore, the entry point for all systems will be that schema along with its published location as a fully qualified path. The meta.xsd serves as a pointer to all required schemata including itself. For example, it points to models.xsd which is a pointer to that part of the dictionary which describes specific models. Interested systems should be pointed to the meta.xsd schema which will, in turn, point to required schemata and metadata of interest. Since the CoreModel Metadata Server is autonomous and automatic, it may change features of the dictionary without notice, so metadata consumers cannot be hard coded to rely on anything other than the published pointer. The entry point schema is subject to change after publication of this documentation, so it is not included and system designers should confer with their local CoreModel system administrator for the current pointer. For guidance in reading and consuming CoreModel's schema based XML, system builders are referred to
To publish the entry point, the system administrator should insure that at least one model is released for publication, configure the server, and then either activate it or test it by pressing the test button. Either event will cause the CoreModel Metadata Server to publish the required schemata as well as the XML metadata. Delivery of the dictionary entry point can then be made by delivering copies of the schema pointer to interested parties. CoreModel maintains dual annotation so that humans can identify the starting point and the links that will be pulled back by their systems.
System Administration The test button will run the server through a single operations cycle. This will test the system and the entered parameters, and will create a complete metadata dictionary based upon the parameters and the data currently in the system. When the test is run, the system may change parameters on the screen, but no changed parameters will be saved unless you press the save button. For example, you may see the activation toggle change from off to on, but the changed parameter will disappear when you close the screen unless you save it. To protect production installations, the system will not allow a test on a computer where the server is already loaded and cycling. If the system detects a loaded server when you press the test button, it will tell you and abort the operation. To run the test, you must either go to a different computer or you must turn the server off and reload CoreModel.
_________________________ CoreReader is a general purpose tool for those who work with data sources. It will query any database in any server on any computer. It has been tested against everything from mainframes to spreadsheets. CoreReader is a true point and click query tool which is designed for operator ease and speed. After the connection is made, the operator points to database objects and operations to perform queries. CoreReader is strongly recommended for modelers and administrators. It allows quick plug-in to access data and metadata in all of an organization's servers and databases. It is a free download from http://CoreReader.com/ .
|
|
|
Technology and web site
|
|
|
Web site is maintained with Notepad.
|
|
