Framework Modeling Guidelines

Cognos 8 Best Practices Transformer Framework Manager Modeling Guidelines
Saravanan Vajjiravel
24-Sept-2011
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
REVISION HISTORY ................................................................................................................................. 3 SUMMARY: ................................................................................................................................................. 4 AUDIENCE: ................................................................................................................................................. 4 INPUTS: ........................................................................................................................................................ 4 DELIVERABLES: ....................................................................................................................................... 4 STANDARDIZE USER INTERACTIONS AND REUSE CALCULATIONS ....................................................... 6 LEVERAGE EXISTING MODELS ............................................................................................................ 6 ENFORCE SOURCE CONTROL ............................................................................................................... 6 EMPLOY A STANDARD NAMING CONVENTION .................................................................................... 6 USE AN APPROPRIATE NUMBER OF MODELS ....................................................................................... 7 DATA SOURCE CONNECTIONS ............................................................................................................. 7 EFFICIENT PROCESSING ....................................................................................................................... 8 LIMITED FUNCTION SET ...................................................................................................................... 8 LINKING AND SEGMENTING ................................................................................................................. 8 VALIDATE THE MODEL.......................................................................................................................15 MODEL MODIFICATIONS ....................................................................................................................15 SYNCHRONIZE THE PROJECT ..............................................................................................................16 TASK AUTOMATION ...........................................................................................................................16 MODEL REPORT .................................................................................................................................16
DESIGN TASKS: ........................................................................................................................................17 BUILD TASKS: ...........................................................................................................................................19 CHECKLIST: ..............................................................................................................................................20 ADDITIONAL REFERENCE DOCUMENTS: .......................................................................................20
Page 2
Revision History Release 1.0 Date 01/06/2011 By Saravanan Vajjiravel Description Base Version
Page 3
Summary: This document is intended to provide guidelines and processes that should be used when implementing Cognos 8 models. Framework Manager (FM) is a desktop tool used to model the metadata for all ad-hoc and standard SQL reporting in Cognos8. Framework Manager is also used to create IQD files that feed PowerPlay Transformer. The information within this document applies to versions Cognos 8.x.x, and should be used in conjunction with the standard Cognos documentation provided with Framework Manager. Be aware that the metadata will have a significant influence on the queries that are generated. To ensure that predictable queries and results are returned, it is necessary to correctly model the metadata. Audience: Cognos Developer (Framework Manager Modeler), Cognos Center of Excellence (Cognos Application Architect) Inputs: Approved Project Detailed Report Specifications Document (to include Service Level/Business Requirement expectations) Data Model Naming Standards as Approved by Data Governance Source to Target Map Deliverables: Framework Manager Model Store deliverables within corporate source control or at a minimum on a shared network drive that is part of a formal back up process Naming conventions for deliverables Tools: Cognos 8 installation on development server Framework Manager installed on developer workstation (Please note: The version must match the version installed on the Cognos 8 Server.) Database connectivity software (Oracle Client, SQL Client, etc.) for all required data sources Access to the development gateway server (your network admin must make provisions to have you set up within the Corporate LDAP and the Cognos enterprise environment.)
Page 4
Overall Considerations: Framework Manager is the foundation to a successful Cognos 8 application. Fully establish user requirements prior to building the Framework Manager model. Listed below are several items that should be taken into consideration before modeling efforts commence: Will this be a high availability system that requires user access on 7x24 basis? How will database updates be handled - perhaps synonyms or a database alias can be used to ensure that source data is always available. What is the expectation for query performance?
Note: This is not the same as report performance and should have been established as part of the project approval process.
Will this application be integrated with another application? How will data security be controlled?
Examples: Row level security, database signons, objects access through the interface.
Will the project support authoring across all of the studios? If the project requires delivery of ad-hoc functionality, then extra effort must be made to ensure that users are presented information in a meaningful way, and appropriate precautions have been taken to prevent run-away, poor performing queries. Consideration may be given to the following options: o Incorporate a prompt against an indexed value in the query subject. The idea is to incorporate a filtering mechanism that prevents the ad-hoc user from performing queries that will perform full table scans or transferring large amounts of extraneous data from the data source to the report server for processing. Several of these could be created (recommend one for every key) and listed as separate Query Subjects. Each of the Query Subject contains all of the columns required for ad-hoc processing and the only difference being the filter applied. Set up a new user connection to the database and put governance on the data source that limits the returned result sets. Governors could also be placed on the Cognos8 side to limit returned output. If summary data is required, create summarized material views to support common access paths.
Does an FM model exist that contains the desired metadata? What are the data sources? Are the data sources using conformed dimensions?
Page 5
Recommended Practices: The Design and Build Tasks outline the specific steps to develop a model; however, execution of these steps should be completed using the principals below: Standardize User Interactions and Reuse Calculations As early as possible in the process, standardize naming conventions, identify filters, and determine query items that will be commonly used as prompts. Calculations that are used across multiple report deliverables should be modeled within Framework Manager. Caution: Overuse of this technique could result in excessively long report specifications and/or poor report performance. Recommendation: Work closely with the appropriate DBA to ensure that the data sources are optimized for reporting with necessary columns to be indexed. Design Approach Determine the modeling approach: relational versus dimensional. Recommended: Sketch out the model on paper before entering the model into Framework Manager and identify all ambiguous joins and data blind spots. (A blind spot is where a join is ignored based on the selected query data.) Reminder: This is not an entity relationship model (ERM): The goal is to provide effective queries, presented in a business-centric format. Leverage Existing Models A high return on investment can be gained for FM work if your firm capitalizes on previous modeling efforts. One of the best ways to accomplish this is by developing a Common Dimension Model that incorporates calculations and filters that are common across the firm. Developers should check with the Cognos Competency Center to validate whether or not the models already exist for their functional areas. Enforce Source Control When starting project make sure appropriate precautions have been taken to protect the work. All files located within the model project folders should be source controlled and checked in at least once a day. Employ a Standard Naming Convention Prior to creating a data source within Framework Manager, determine an appropriate naming convention. The model source should be named in a manner that clearly defines the application and packages should be named with the application and business functional area to be deployed. Once project approval has been granted, the Cognos Center of Excellence will provide a prefix that should be used for all model and package source.
Page 6
Use an Appropriate Number of Models As a rule, there will be one model created for a single application. However, the following factors should be considered when determining whether to create a new model:
o o o o
The redundancy of information when compared to currently developed model. The number of tables and attributes. Consistency in business logic and presentation. Size of currently deployed models.
Data Source Connections Once the appropriate name has been determined, submit a request to the Cognos Center of Excellence to create the data source connection within Cognos Connection. If the data source information does not exist within Cognos Connection, the metadata will not be available to the Framework Manager modeler. Below are options to ensure that the data source is recognizable regardless of the deployed environment: o
Create a generic name such as DB2 that could be used regardless of the
environment. The client connection is configured to the appropriate environment such as DB2Test, DB2QA, DB2Prod.
Create a parameter map for the catalog and schema name. This macro will read defaultName session parameter and will substitute the appropriate variables at run
time. Choosing this option gives you the greatest flexibility for deployment.
Syntax for the Data Source Catalog parameter map:

#$Datasource_Catalog {$account.defaultName}#
Syntax for the Data Source Schema parameter map:

#$Datasource_Schema {$account.defaultName}#
It is also possible to specify a connection string to an XML data source to include a parameterized XML. However, using parameterized XML introduces the following restrictions: o o o If the URL component is a prompt, it cannot contain other data. Data cannot be imported through Framework Manager Query validation does not work through Report Studio
Page 7
Efficient Processing Determine if all data processing can be accomplished at the database level. Abstraction of data processing to the report application server level typically results in slower processing and higher resource consumption on the report server(s). A decision to perform local processing should be based on the following: o o A clearly identifiable need for specific processing not available from the database such as cross database joins and unsupported SQL99 functions. Anticipated data volume in the production environment.
Limited Function Set Limit the function list in accordance with the native source.
Linking and Segmenting Recommended: Linking and segmenting are recommended for large models. These techniques are also valuable when multiple FM developers are required. o Links are created to help organize work across large projects, to maintain consistency, and to reuse information. For example, the project named Inventory contains the folder named Products. You can create a link from the Sales Products to Inventory Products. If any changes or additions are made to the Inventory Products folder, you will see them in the Sales Products folder. A linked project is shared by other projects. It should not be created in a subdirectory within the master project directory. o A segment is a project within a main project. A segment is owned by its main project. A project segment is a complete project and changes to that project impact all projects to which it is linked. If you want to open a segment as a separate project, it must be structured as a complete project. There must be a physical layer in each segment that contains a subset of the data source query subjects on which they are based. These data source query subjects provide access to the data and metadata and must be included in the appropriate segments. Note: When changing the project structure, do not open the segments as individual projects. Instead, check the main project and make changes from within it. Tips: Do not change the physical layer in a segment. Any change will be reflected in the linked parent model and will impact all model segments that share data source query subjects. Changes may not be apparent outside the model in which they are made until the model is closed and reopened. Before a project is segmented, ensure that the folder and namespace are named correctly. You cannot rename the folder or namespace after it has been segmented.
Page 8
Create Multiple Layers At a minimum, a data layer and presentation layer should be created. Recommended: If you are modeling multiple fact tables that are joined to common dimensional information, create multiple namespaces under the presentation layer. A typical look & feel of FM project explorer;
Page 9
Layer 1 - Data: The data layer will consist of the content of those tables required to process the work. If effective data warehousing modeling techniques are employed, most of the data may be retrieved from the tables. However, if there is data that will NEVER be used, then include a filter for the query subject. For the off-clarity, organize the tables by name (to include alias tables).
1. Detecting Cardinality from the Data Source When importing from a relational data source, cardinality is detected based on a set of rules that you specify. The available options are: Use primary and foreign keys. Use matching query item names that represent uniquely indexed columns. Use matching query item names. The most common situation is to use primary and foreign keys as well as matching query items that represent uniquely indexed columns. The information is used to set some properties of query items as well as to generate relationships. To view the index and key information that was imported, right-click a query subject and click Edit Definition. 2. Create relationships and ensure cardinality rules are appropriate. This is an extremely important component of the modeling process. Framework Manager uses the cardinality rules to assist the query engine in generating the appropriate SQL statements. Different types of cardinalities are: 1. One-to-one (1..1) - One-to-one (1..1) relationships occur when one instance of data in a query subject relates to exactly one instance of another. For example, each student has one student number. 2. One-to-many (1..n) or zero-to-many (0..n) relationships occur when one instance of data in a query subject relates to many instances of another. For example, each teacher has many students. 3. Many-to-many (1..n) - Many-to-many (n..n) relationships occur when many instances of data in a query subject relate to many instances of another. For example, many students have many teachers. Use 1..n or 0..1 for dimensionally aware queries Identifies fact tables in star schemas Identifies multi-fact queries and generated stitched SQL statements. Use 1..1 or 0..1 for default intersection queries (Non-stitched SQL).
Page 10
Screenshot reference
3. Do not create fact-to-fact joins as these can create problems (such as query performance due to data volume, query performance with outer joins, different grains of detail). Fact tables should be separated by a dimension. 4. Validate that keys and other identifiers are correctly identified. This is important because as automatic aggregation occurs for numeric data that may really be an attribute as opposed to a fact. Tip: Make sure the fact data that will be used in calculations is set to properly handle null values. 5. Resolve loop joins or ambiguous relationships. The most common type of ambiguous relationships are where multiple valid relationships exist between two tables, reflexive relationships (table joins to itself). This can be resolved by creation of alias tables; however, it is not recommended to build deep hierarchies to resolve reflexive relationships. This should be accomplished by flattening out the table. 6. Create star schema groupings when they are opportunities to build multiple SQL paths or there is a need to create factless queries. 7. Set the SQL Generation type. (Minimized means that the generated SQL contains the minimum number of tables and joins required to obtain values for the selected query items.)
Page 11
8. Validate the data being returned in the Query Subject. Tip 1: Remember that calculated values are treated as facts. As such, these values may also need to be tested outside of the Framework Manager Query Subject. Recommend testing through one of the studios. Tip 2: Frequently, development is performed against a database that represents a very small percentage of the data that will later be queried. During the project, be sure to test against data that is representative of production environment data. Tip 3: Make sure that fact data that will be used in calculations is set to properly handle null values. 9. If row-level data security has been applied using session variables, test by overriding the session parameters. 10. Use parameterized SQL to provide dynamic filtering and grouping capability to the report authors. 11. Add dimensional information to the Query Subject if multiple levels exist within a single physical table. For instance, if a table has been de-normalized to contain both the product and product type information (not just the key) data. Levels should be set to ensure Framework Manager understands the appropriate grouping algorithms. 12. Use determinants to identify unique data within a de-normalized table. Determinants reflect granularity by representing subsets or groups of data in a query subject. They are used to ensure correct aggregation of this repeated data. Determinants are most closely related to the concept of keys and indexes in the data source; they are imported based on unique key and index information in the data source. An example of a unique determinant is Day in Time dimension. An example of a non unique determinant is Month; the key in Month is repeated for the number of days in a particular month. When you define a non-unique determinant, you should specify group by. The group by indicates that when the determinants keys or attributes are repeated in the data, Cognos 8 should apply aggregate functions and grouping to avoid double-counting. If you have multiple determinants with Uniquely Identified check box selected, only the Group By setting of the first determinant is used. This is because the processing of determinants stops at the first determinant with Uniquely Identified check box selected. Below example illustrates some determinants and how you would define them as either unique or non-unique with a group-by. Year Key 2006 2006 Month Key 200601 200601 Month Name January 06 January 06 Day Key 20060101 20060102 Day Name Sunday January 1, 2006 Monday January 2, 2006
For this data set, you can define three determinants; two group-by determinants (Year and Month) and one unique determinant (Day).
Page 12
Determinants and Attributes If a determinant defines attributes, all query items in the query subject must be associated with the determinant and defined as either a key or an attribute. You can have a determinant with no attributes defined for it. Framework Manager uses this type of determinant to indicate which query items are indexed. In the snapshot below, you can see 3 determinants for year, month and day created for the Time Dimension
Page 13
When to Use Determinants While determinants can be used to solve a variety of problems related to data granularity, you should always use them in the following primary cases: o A query subject that behaves as a dimension has multiple levels of granularity and will be joined on different sets of keys to fact data. For example, Time has multiple levels, and it is joined to Inventory Fact on the Month Key and to Sales Fact on the Day Key. There is a need to count or perform other aggregate functions on a key or attribute that is repeated. For example, Time has a Month Key and an attribute, Days in the month that is repeated for each day. If you want to use Days in the month in a report, you do not want the sum of Days in the month for each day in the month. Instead, you want the unique value of Days in the month for the chosen Month Key. In SQL, that is XMIN (Days in the month for Month Key).
13. Collapse hierarchical information into a single Query Subject. 14. Avoid incorporating query subjects that act as both facts and dimensions. 15. By default, Cognos 8 will aggregate all query items that are identified as a fact so ensure that numeric items are properly identified.
Page 14
Layer 2 - Presentation The arrangement of the presentation layer will be largely driven by the required deliverables. 1. Create a layer for the Report Studio developers. In creating the developer view, include the primary content from the data view. This provides the greatest flexibility for report authors. They can look up reference information, validate their data, create calculations, etc. Within this layer, create additional calculations, commonly used data filters , and (as appropriate) additional query subjects that may be used for more complex authoring needs. Tip 1: Where a filter will be used exclusively against a query subject, test it through Report/Query Studio to ensure that you are achieving the desired results. Tip 2: If possible, use shortcuts. 2. Create a layer for Query Studio. This will be a simplified view intended for case-by-case use by the end user. The layer intended for report authors will not be visible in Query Studio. This layer must be organized and presented in a manner that is easily understood. Create a Package for Each Functional Area Recommended: Create a separate package for each functional area. This ensures that the package stays as trim as possible and limits the demand for regression testing. Coordinate the publication of all packages with the Cognos Center of Excellence to ensure appropriate access has been granted. Validate the Model During the development of the Data Model, the author should be testing their query subjects to ensure that the appropriate data is being returned. Always validate the project before publishing to the server. The tool requires that all errors be repaired prior to publication; however, warnings should be closely examined to ensure the future integrity of the model. Tip: Development is frequently performed against a database that represents a very small percentage production data. At some point in time during the project, these tests should be performed against data that is more representative of the production environment. Model Modifications Once the report authors begin development, modifications to the structure of the Framework Manager model may negatively impact the report authoring process. Recommended: Through the Framework Manager model, find all report dependencies. Create a validation package using the Cognos Center of Excellence assigned naming convention. The report authors will be notified that changes have been published to the validation package and the dependent reports should be tested against the validation package. After the successful validation of the reports, the report authors will notify the Framework Manager that the package can be republished. Tip: Ensure that only one copy of the validation package is saved to Content Manager.
Page 15
Synchronize the Project Frequently, changes are made to the underlying data sources that need to be propagated to the model. The preferred method of doing this is to synchronize the project. However, if the project is large and complex, this may be fairly time-consuming. You may choose to update individual query subjects, by selecting the update option under the tools menu.
Task Automation If there is a need to perform identical tasks against multiple models, perform repetitive tasks or fully reconstitute a model, convert the log files to scripts to perform repetitive tasks or fully reconstitute a new model, save the transaction logs as scripts and execute the script as appropriate.
Model Report Create a model report to serve as modeling artifact.
Page 16
Design Tasks:
1. Naming Conventions a. Propose Data Source Naming Convention b. Propose Model Naming Conventions c. Propose Model Namespace and Folder Naming Conventions Folders are used to organize objects by subject or functional area. This makes it easier for you to locate metadata, particularly in large projects. Drawback: The main drawback of folders is that they require unique names for all query subjects, dimensions, and shortcuts. Therefore, they are not ideal for containing shared objects such as conformed dimensions. d. Propose Query Subject/Query Item Naming Conventions 2. Data Preparation a. Identify Source Data b. Identify Method of Accessing Data (e.g., native, ODBC) 3. Model Design a. Identify location to be used for repository control b. Design Model Namespace/Folder Structure (e.g., star, transaction) c. Design Model Join Strategies d. Design Model Filters e. Design Parameter Maps f. Design Model Calculations g. Develop Package Strategy h. Identify Alias & Synonyms Issues
Page 17
i.
Identify attribute usage Usage property identifies the intended use for the data represented by each query item. During importing, the Usage property is set according to the type of data that the query items represent in the data source. You need to verify that this property is set correctly. For example, if you import a numeric column that participates in a relationship, the Usage property is set to identifier. You can change the property. For relational query items, the value of the Usage property depends on the type of database object the query item is based on. Usage Property Identifier Database Object key, index, date, datetime Description Represents a column that is used to group or summarize the data in a Fact column with which it has a relationship. Also represents an indexed column. Also represents a column that is of the date or time type. Represents a column that contains numeric data that Represents a column that is neither an identifier not fact such has descriptions
Fact Attribute
numeric, time interval String
j.
Identify formatting requirements
4. Validate, Create, and Review Prototype Model 5. Deliver Proposed Physical Cognos8 Data Flow Schematic 6. Deliver Proposed Model Table, Join, & Namespace/Folder Document
Page 18
Build Tasks: 1. Create Physical View a. Create Data source b. Add Tables c. Setup Joins d. Alias Tables (Synonyms) e. Modify attribute usage (i.e. differentiate attribute, facts and identifiers) f. Modify formatting as needed ($, % ) g. Analyze Joins (minimize outer joins for performance and functionality) h. Setup Namespaces, Folders & Query Subjects i. Identify Table Filters j. Test Joins 2. Create Performance Views a. Resolve ambiguous joins b. Set dimensional information 3. Create Business Views (Phase 1 Simple) a. Organize Folders & Query Subjects (use Star Schema Groupings when possible) b. Rename Columns with Business Names c. Validate Demo Phase 1 Package (Usability Test) 4. Create User Views (Phase 2 - Enhanced) a. Re-Organize Folders & Query Subjects b. Adjust Columns Based on Feedback c. Construct Model Calculations d. Construct Conditions e. Validate Demo Phase 2 Package (Usability Test) 5. Create User Views (Phase 3 - Optimized) a. Finalize Folder, Query Subject, Calculation, Filter Changes b. Test Joins With Representative Ad-Hoc Queries c. Validate Demo Phase 3 Package (Usability Test) 6. Apply Security to Model a. Create Object Security b. Create Data Security 7. Cognos Connection a. Model Packaging & Publishing b. Testing i. Security ii. Package Functionality c. Load Testing i. Database ii. Server Performance iii. Network Communications iv. Web v. Determine Critical Mass
Page 19
Checklist:
Complete model design Complete model testing (including functional and performance unit testing) Publish model and apply security to package, objects, and data Test Query Subjects changing sessions parameters Perform a Checkpoint Review with appropriate Cognos Center of representative Create FM artifacts
Excellence
Additional Reference Documents:

1. Framework Manager User Guide (i.e. shipped along-with Cognos8 FM installation) 2. Framework Manager Best Practices
For your quick reference, please find below embedded document
Framework_Manager _Best_Practices.pdf
Page 20

Framework Modeling Guidelines

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Framework Modeling Guidelines

Caricato da

Copyright:

Formati disponibili

Cognos 8 Best Practices Transformer Framework Manager Modeling Guidelines

Syntax for the Data Source Catalog parameter map:

Syntax for the Data Source Schema parameter map:

Model Report Create a model report to serve as modeling artifact.

numeric, time interval String

Identify formatting requirements

Additional Reference Documents:

Potrebbero piacerti anche