Moss L.T., Atre S. Business intelligence roadmap: The complete project lifecycle for decision-support applications
Подождите немного. Документ загружается.
Meta data can be centrally managed, or it can be distributed. Either way, each
instance of a meta data component should be unique, regardless of its physical
location.
Meta Data Categories
There are two categories of meta data: business meta data and technical meta data.
Business meta data provides business people with a roadmap for accessing
the business data in the BI decision-support environment. Since many business
people are relatively nontechnical, they should have access to meta data, which
defines the BI decision-support environment in business terms they understand.
1.
Technical meta data supports the technicians and "power users" by providing
them with information about their applications and databases, which they need
in order to maintain the BI applications.
2.
Table 7.1 highlights some differences between business meta data and technical
meta data.
Table 7.1. Business Meta Data versus Technical Meta Data
Business Meta Data
Technical Meta Data
Provided by business people
Provided by technicians or tools
Documented in business
terms on data models and in
data dictionaries
Documented in technical terms in
databases, files, programs, and tools
Used by business people
Used by technicians, "power users," data-
bases, programs, and tools (e.g., ETL,
OLAP)
Names fully spelled out in
business language
Abbreviated names with special characters,
such as "_" (underscore) or "–" (dash),
used in databases, files, and programs
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
231/631
[ Team LiB ]
Meta Data Repository as Navigation Tool
Meta data is not new; it has always been part of operational systems. It can be found
in systems documentation, record layouts, database catalogs, and data declaration
sections in programs. The role of meta data in an operational environment was
always viewed as systems documentation, which was mainly used by the technicians
who maintained the operational systems. When some of the systems documentation
(meta data) became outdated, the technical staff had enough skills to read through
the actual programming code to extract the information they were looking for, such
as the meaning and content of a data element. Thus, more often than not, meta data
was treated as an afterthought.
In a BI decision-support environment, meta data takes on a new level of importance.
A new audience has to be serviced, namely, the business people. Meta data helps
them locate, manage, understand, and use the data in the BI target databases. Meta
data has a new role: navigation, not just documentation. Business people ordinarily
do not have the technical skills, nor the time or desire, to decipher programming
code. They also do not want to stay dependent on the IT department to interpret the
meaning and content of the data after it has been manipulated by the programs.
Rather than calling a programmer, a business person should be able to access the
meta data, which would then help him or her effectively navigate through the BI
decision-support environment and interpret the BI data. As illustrated in Figure 7.1,
meta data describes what data is available in which BI target database, where the
data came from, how to access it, how to drill down to the detailed data for closer
examination, and how to use it.
Figure 7.1. Using a Meta Data Repository as a Navigation Tool
Data Standardization
If business data had been stored and used in a consistent, approved manner all
along, the data redundancy and inconsistency problems that currently plague many
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
232/631
operational systems would not exist to the extent they do today. Unfortunately, bad
habits die hard. Developers and business people still explicitly or implicitly reuse the
business data in operational systems for different purposes. For example, developers
still explicitly redefine data elements in their programs, and business people still
implicitly redefine (invent new codes for) existing data elements to capture unrelated
information. Documentation of these redefinitions also remains poor or nonexistent.
If any documentation exists, it is rarely distributed to everyone in the organization
who needs it, and it is very seldom kept up-to-date. Therefore, business people
continue to invent their own business rules and create their own redundant data
along with redundant processes.
Every BI project team must address this existing data chaos and must make every
attempt to promote the standardization of data. While standardizing the business
data for the BI decision-support environment, the BI project team should document
all changes made to the data so that everyone can be aware of them. This
documentation takes the form of meta data in the meta data repository. For
example, a source data element could be renamed to conform to new naming
standards, or data values could be filtered, added, or transformed to enforce a
business rule. In both cases, the BI data in the BI target database would no longer
match the source data in the source file or source database. The meta data would
provide the navigation between the two.
Using BI applications without knowing that the business data was changed and how
it was changed can be a frustrating experience that can eventually end with the
business people no longer wanting to use the BI applications at all. That would be
devastating since one of the most important aspects of a BI decision-support
initiative is to provide an easy-to-use, intuitive way for the business people to access
and query the data. An easy-to-use application means the business people:
Have no need to be relational technology experts
Have no need to know Structured Query Language (SQL)
Have no need to know the physical structure of the databases
Have no need to know the location of their data
Have no need to guess the meaning of the data
Have no need to search for the required information
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
233/631
[ Team LiB ]
Meta Data Classifications
Since BI projects can generate a great number of meta data components, it is useful
to classify these components and to prioritize them for incremental implementation.
Groupings of Meta Data Components
Meta data components can be sorted into four meta data groupings or classifications:
ownership, descriptive characteristics, rules and policies, and physical characteristics
(Figure 7.2). The meta data repository should be able to store the meta data
components of all four classifications, as listed below.
Figure 7.2. Meta Data Classifications
Ownership
Data owner: Data is owned by the organization. However, since the
organization is a legal entity and not a person, someone in the organization
must take on the authority and responsibility to set policy, determine rules, and
establish standards for the organizational data. This authority and responsibility
can be distributed among line-of-business managers or assigned to a data
ownership committee (whose members will most likely be some or all of the
line-of-business managers). An example of distributed data ownership is a
manager of the human resource department who has the authority and
responsibility to establish policies, rules, and standards for payroll data but not
for product data. With data ownership by committee, the committee establishes
policies, rules, and standards for all data by consensus, by delegation to a
committee member, or by some other committee rule.
Application owner: Traditionally, ownership has been assigned to a system as
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
234/631
a whole. Since a system is usually composed of an application and its data,
"system ownership" implies that the same person has authority to set policy,
determine rules, and establish standards for both data and functionality (the
application). That may be a valid condition for operational systems where data
is originated, but it is not valid for BI applications because most business
people using the BI applications are not the same individuals who originate the
operational data. Therefore, BI information consumers may own the BI
application, but most of them will not own the data.
Descriptive Characteristics
Name: Every data object, data element, and business process should have a
unique name.
Definition: Every data object, data element, and business process should have
a brief definition explaining what it is.
Type and length: Every data element should have an official type and length
declared for it, even if the data elements in the source systems or the columns
or cells on the target databases may deviate from it. That deviation would also
be defined as meta data under the data element, the column, or the cell where
it occurred.
Domain: Every data element should have a declared set of allowable values,
even if the set is all-inclusive, such as "any character, number, or sign."
Notes: Additional facts of interest about data or processes should be included.
This is a catchall for free-form comments, such as "Dispute between
engineering and marketing regarding the meaning of Product Subcomponent
Type Code was turned over to the BI steering committee for resolution."
Rules and Policies
Relationships: Data objects are related to each other through business
activities. The meta data repository should be able to store information about
these relationships.
Business rules and business policies: These components can apply to data
as well as to processes. They can be technical data conversion rules, business
data domain rules, business data integrity rules, or processing rules.
Security: Requirements for security can apply to data, meta data, processes,
databases, applications (programs and screens), tools, and Web sites.
Cleanliness: Metrics about the ETL reconciliation totals and about the quality
of the BI data should be stored. The metrics can be expressed as reliability
percentages of a data load (e.g., 89 percent of the customer type code is valid)
or as record counts stating the number of records filtered (rejected) and the
number of records passed through during the ETL process.
Applicability: Data does not live forever. Occasionally, new data is invented
and captured, and old data is retired and no longer used. Since the BI target
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
235/631
databases store many years of history, some columns or cells will not have
values for all time periods because the data was not applicable or did not exist
during certain time periods. If spikes appear on trend analysis graphs, the meta
data repository should be consulted to determine the applicability of that
particular piece of data.
Timeliness: Business people will want to know when the source data was last
updated and which of the versions of the operational systems were used for the
update. Not all operational systems run daily or on the same day of the month.
One operational system may run on the last calendar day of the month while
another may run on the last business day of the month. Some operational
systems do not "close out the month" until they complete an adjustment run
four to ten days after the last calendar day of the month.
Physical Characteristics
Origin (source): Since BI target databases only store existing operational data
(internally generated and externally purchased), the origin or source for each
data element should be documented. One column in the BI target database can
be populated with data elements from multiple sources. For example, the
column Account Balance in the Account table could be populated from the data
element Demand Deposit Account Balance in the Checking Account source
database and from the data element Time Deposit Account Daily Balance in the
Savings Account Transaction file. Conversely, one source data element can feed
multiple columns in the BI target database. For example, the data element
Type Code may be used for two purposes in the operational system. The data
values "A", "B", and "C" of Type Code may be used to populate the column
Customer Type Code in the Customer table, and the data values "N", "O", and
"P" of the same Type Code may be used to populate the column Product Type
Code in the Product table.
Physical location: Several meta data components (e.g., tables, columns,
dataset names) should describe where the data resides in the BI decision-
support environment.
Transformation: Very few data elements can be moved from source to target
without any type of transformation. At a minimum, the data type and length
may have to change, or single-character codes may have to be translated into
multi-character mnemonics. In the worst case, lengthy business rules may
require more complicated transformations involving editing, filtering,
combining, separating, or translating data values.
Derivation: This component stores the calculation for derived columns. While
derived columns are customarily not stored in operational systems, it is the
norm to store them in BI target databases.
Aggregation and summarization: Similar to derivation, aggregation and
summarization rules should be stored as meta data.
Volume and growth: The size and growth of BI target databases are often
enormous. Therefore, projected as well as actual volumes should be
documented as meta data in terms of the number of rows and the percentage of
expected growth.
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
236/631
Business people most frequently access the meta data components in the descriptive
characteristics classification as well as the rules and policies classification (Figure
7.3). Technicians typically access the meta data components in the physical
characteristics classification (Figure 7.4).
Figure 7.3. Meta Data Usage by Business People
Figure 7.4. Meta Data Usage by Technicians
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
237/631
Prioritization of Meta Data Components
Capturing all meta data components may not be necessary or practical for all BI
projects. However, capturing none is unacceptable. As a rule, meta data should be a
deliverable with every BI project. It will serve the business people to recognize their
old data, trace what happened to it (transformation), locate it in the new BI target
databases, and determine how to use it properly. In other words, the business
people will greatly benefit from having meta data available to help them navigate
through the BI decision-support environment.
Not all meta data components have the same value to all business people or all BI
applications. It might be useful to prioritize the meta data components into three
groups: mandatory, important (beneficial but not mandatory), and optional. Table
7.2 shows a recommended prioritization scheme for capturing meta data components
in a meta data repository.
Table 7.2. Prioritization of Meta Data Components
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
238/631
Meta Data
Mandatory
Important
Optional
Owner
Business data name
Technical data name
Definition
Type and length
Content (domain)
Relationships
Business rules and policies
Security
Cleanliness
Applicability
Timeliness
Origin (source)
Physical location (BI databases)
Transformation
Derivation
Aggregation
Summarization
Volume and growth
Notes
All mandatory meta data components, and as many
important meta data components as possible, should be
captured and stored in the meta data repository. Optional
meta data components could be postponed to future BI
application releases.
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
239/631
[ Team LiB ]
Meta Data Repository Challenges
Good ideas are often hard to implement. Providing a meta data repository is a good
idea but also quite a challenging one, regardless of whether the decision is made to
license (buy) a commercially available product or to build a repository from scratch.
This section briefly describes the challenges faced when implementing a meta data
repository (Figure 7.5).
Figure 7.5. Meta Data Repository Challenges
Technical Challenges
Building a meta data repository is not a trivial task. It is a project in itself, with its
own project plan, its own development steps, and its own staff. All the technology
challenges that apply to databases and applications can surface on meta data
repository projects.
Licensing a meta data repository product is an alternative to building one, but the
"plain vanilla" versions of commercially available meta data repository products
often do not meet all the meta data requirements of a BI decision-support
environment. Therefore, licensing a meta data repository product still necessitates
extensive analysis of the requirements in order to select the right product, as well as
a considerable implementation effort to enhance it.
Enhancing licensed software comes with its own challenges. The source code for the
product may not be available. The vendor may insist on incorporating the requested
enhancements for a price and at his or her own speed. The time and effort required
for product maintenance increase because the enhancements must be reapplied to
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
240/631