Moss L.T., Atre S. Business intelligence roadmap: The complete project lifecycle for decision-support applications

Подождите немного. Документ загружается.

2. Test the ETL tool

functions

Review the

transformation

specifications in the

source-to-target

mapping document

Determine whether the

ETL tool functions can

perform the required

transformation logic

Determine what

supplementary custom

code must be written for

transformations that

cannot be handled by the

ETL tool

3. Design the ETL process

flow

Determine the most

efficient sequence in

which source data can be

extracted from the

source files and source

databases

Determine the most

efficient sequence in

which the extracted

source data can be

transformed, cleansed,

and loaded

Determine the sort and

merge steps in the ETL

process

Identify all temporary

and permanent work files

and tables

Determine what

components of the ETL

process can run in

parallel

Determine what tables

can be loaded in parallel

Draw the process flow

diagram showing process

sequence and process

dependencies for:

- Extracts from

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

531/631

source files and

source databases

- Temporary and

permanent work

files and tables

- Sorting and

merging performed

on the temporary

and permanent

work files and

tables

- Transformation

programs

(program modules)

- Error rejection

files and error

reports

- Load files and

load utilities

4. Design the ETL

programs

Design three sets of ETL

programs for:

- Initial load

- Historical load

- Incremental load

Modularize the ETL

programs as much as

possible

Translate the

transformation

specifications from the

source-to-target

mapping document into

programming

specifications for each

ETL program module (or

for each ETL tool

module)

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

532/631

5. Set up the ETL staging

area

Determine whether the

entire ETL process can

run in one central

staging area or whether

it has to be distributed

(some ETL programs

running on the

mainframe, some on the

ETL server)

Set up the ETL server (if

a dedicated server is

used)

Allocate space for

temporary and

permanent work files and

tables

Create program libraries

Establish program-

versioning procedures

10. Meta Data

Repository Design

1. Design the meta data

repository database

Review the logical meta

model for the meta data

repository

Design the meta data

repository database

(entity-relationship or

object-oriented)

Draw the physical meta

model diagram (entity-

relationship or object-

oriented)

Map the physical meta

model to the logical meta

model

Create the DDL for the

meta data repository

database

Create the DCL for the

meta data repository

database

Design backup and

recovery procedures

Design versioning and

archival procedures

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

533/631

2. Install and test the

meta data repository

product

Compile a list of meta

data repository products

and vendors

Compare the meta data

repository products to

the meta data repository

requirements in the

revised application

requirements document

and in the logical meta

model

Create a scorecard for

each evaluated meta

data repository product

Create a scorecard for

each evaluated meta

data repository vendor

Narrow the list of meta

data repository products

and vendors to a short

list

Arrange for meta data

repository product demos

Check the vendors' client

references

License (buy) the meta

data repository product

Install and test the meta

data repository product

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

534/631

3. Design the meta data

migration process

Analyze all sources for

extracting business meta

data (for example, CASE

tool, word processing

documents,

spreadsheets)

Analyze all sources for

extracting technical meta

data (for example, ETL

tool, OLAP tool, data

mining tool)

Design the tool interface

process

Design the

transformations for the

extracted meta data

Design the load

programs for the meta

data repository

Write the programming

specifications for the

meta data migration

process:

- Tool interface

process

- Transformation

process

- Load process

4. Design the meta data

application

Design the meta data

repository report

programs

Design the context-

sensitive online help

function

Design the media for

displaying meta data ad

hoc query results (for

example, PDF, HTML)

Design the access

interface process (Web

display, GUI front end,

meta data repository

directory)

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

535/631

Write the programming

specifications for the

meta data application:

- Reports

- Queries

- Access interface

process

- Online help

function

11.

Extract/Transform/Load

Development

1. Build and unit test the

ETL process

Code the ETL programs

by following the

programming

specifications in the ETL

program design

document

If using an ETL tool,

write instructions

(technical meta data) for

the ETL tool modules

Capture the ETL technical

meta data for the meta

data repository

Write code in the ETL

programs to produce

reconciliation totals, data

quality metrics, and load

statistics

Unit test each individual

program module

If using an ETL tool, unit

test each ETL tool

module

Write the scripts to

execute the ETL

programs and the sort,

merge, and load utilities

in the proper sequence

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

536/631

2. Integration or

regression test the ETL

process

Create a test plan with

test cases for the ETL

process

Create test data (a

representative subset of

source data) for the ETL

programs

Integration or regression

test the entire ETL

process from beginning

to end using the test plan

Log the actual test

results and document

any test issues

Compare actual test

results with expected test

results

Revise the ETL programs

(or the instructions for

the ETL tool)

Retest the entire ETL

process from beginning

to end until it produces

the expected results

3. Performance test the

ETL process

Test individual ETL

programs and ETL tool

modules that read or

write to high-volume

tables

Test the parallel

execution of ETL

programs and ETL tool

modules against high-

volume tables

Test the ETL programs

and ETL tool modules

that perform complicated

operations

Use full-volume data for

performance testing

If using a stress test

simulation tool, define

the test components

(programs, databases,

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

537/631

tables, volumes, and so

on) to the simulation tool

and run a simulation test

before testing with full-

volume data

4. Quality assurance (QA)

test the ETL process

Move all ETL programs

into the QA environment

QA test the entire ETL

process from beginning

to end with the

operations staff

Obtain approval from the

operations staff to move

the ETL process into

production

5. Acceptance test the ETL

process

Acceptance test the

entire ETL process from

beginning to end with the

subject matter expert

and the business

representative:

- Validate all

cleansing

transformations

- Validate error-

handling routines

- Validate

reconciliation totals

Obtain certification for

the ETL process from the

business representative

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

538/631

12. Application

Development

1. Determine the final

project requirements

Review the results of the

prototype

Review the prototyping

programs and scripts

Review the change-

control document

Review the issues log

Review existing and

mock-up report layouts

Review existing

spreadsheets

Review the latest version

of the application

requirements document

Agree on the final project

requirements;

renegotiate the final

scope if necessary

Update the application

requirements document

to reflect any changes

2. Design the application

programs

Design the final reports

Design the final queries

Design the front-end

interface (GUI, Web

portal)

Design the online help

function

Write the programming

specifications for:

- Reports

- Queries

- Front-end

interface process

- Online help

function

Create a test plan with

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

539/631

test cases and a test log

3. Build and unit test the

application programs

Create sample test data

Load the development

databases with sample

test data

Rewrite or enhance

prototyping programs

and scripts

Code the final report

programs

Code the final query

scripts

Code the final front-end

interface programs

Code the online help

function programs

Unit test each individual

program module

4. Test the application

programs

Integration test (first

release) or regression

test (subsequent

releases) all programs

and scripts from

beginning to end, using

the test cases from the

test plan:

- Report programs

- Query scripts

- Front-end

interface programs

- Online help

function programs

Log the actual test

results and document

any test issues

Compare actual test

results with expected test

results

Revise the application

programs and scripts

北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视仅供免费试用/版权原著所有

540/631