PROJECT LESSONS LEARNED REPORT

Project Name:

Business Intelligence (BI)

Prepared by:

Diane Kleinman

Date:

June 15, 2009

Project Close-Out Discussion

A Lessons Learned meeting was held on 6/12/09. The summarized lessons learned survey results are attached to
this document.
Attendees:
Vel Angamthu
Wendy Berkowitz
Wayne Bowker
Aaron Demenge
Michael Garza
Jim Green
George Hardgrove
Janet Heller
Bill Kanfield

Janet Heller
Bill Kanfield
Ann Lundholm
Ron Mapston
Peggy McCarthy
Tammy Nelson
Bill Paulus
Jennifer Pierson
Shari Zeise

List this projects biggest successes.

Description

Factors that Promoted this Success

We have a more organized reporting

structure.

The capabilities of BI allow for a more organized reporting

structure

We were able to remove a lot of

reports that werent being used and
we have reports that actually work.

A benefit of this project was time was taken to examine

existing reports and to remove those reports that no longer
were needed or did not work.

The tool will allow users to write their

own reports and these can be
modified on an ongoing basis

BI allows more than just a few people to have the capability

to create reports. There is no bottleneck like there was
before when only one person knew and was able to write
reports that are needed. Now users can write their own
reports and reports can be modified on an ongoing basis.

List areas of potential improvement along with high-impact improvement strategies:

Category
Project Management

Project Shortcoming
There are still questions around
whether or not to still treat this as a
project and let the team make
decisions on how to move forward
with operational monthly reports.

Project Communication

The right people were not always

included on project teams and
sometimes the teams needed to
change mid-stream as the project
requirements changed.

LESSONS_LEARNED_REPORT BI Project

Lesson Learned
FM Leadership needs to
determine if this project should
continue with a project structure
in place. There needs to be a
focus on completing operational
monthly reports.
Continually monitor the makeup
of project teams to make sure
that the right people are
included when requirements
change.

Page 1

Project Close-Out Discussion

Project Roles &

Responsibilities

Requirements

Testing

Weekly project status reports did

not always accurately reflect the
real-time status of the project.
These reports became more
valuable towards the end of the
project because they more
accurately reflected what was
happening. Part of the reason for
this is that there were sometimes
conflicting reports from project
teams and there were day-to-day
changes in the status of issues.
There were other projects besides
BI that also had priority and
resources were pulled in multiple
directions (e.g., 8iR2, PeopleSoft,
Pillar projects). Team member did
not lack interest or commitment;
there were just competing priorities
that made it difficult to dedicate the
time needed to meet all scheduled
deadlines. As an organization, we
need to figure out what our project
capacity is.
Key decision makers were not
involved early enough.
Requirements from top level
management came in too late.
The end that we had in mind was
different from the bosss end.
Need to formalize the report
request process. Some vetting is
required to do any monthly and ad
hoc reports. Sending an email with
a report request is not enough.
A design document that clearly
defines business rules needs to be
done for any new universe data
(e.g., parts universe).

As new reports/needs were

defined, then the design changed
and that impacted previously
created reports. I felt like I was
working on the beach and every so
often the tide would come in and
wash my work away.

LESSONS_LEARNED_REPORT BI Project

Status reports should be more

direct and reflect what the real
project issues are. This can be
challenging when things are
moving quickly on a project and
the status report is just a
snapshot of what is happening
at the time.

FM Leadership needs to
determine project priorities
across the organization and this
should be clearly communicated
to management so resources
are not pulled in multiple
directions.

Project direction needs to be a

top-down approach with input
from top level management
earlier in the project rather than
later after requirements have
already been defined.
Consider formalizing the request
process for monthly and ad hoc
reports.

A design document needs to be

done before bringing in
someone to do the design work.
This will help eliminate any
confusion and will save time in
the end.
A test environment needs to be
set up early in the project.

Page 2

Project Close-Out Discussion

Training

FM needs to determine how

reports will be used by FM users.

Training plan should be

developed for different user
roles which includes:
- How to navigate in BI
- How to read and
understand data in
reports
- What actions should be
taken with reports

Enter other comments:

Project Lessons-Learned Document / Signatures

Project Manager:

Vel Angamthu

I have reviewed the information contained in this Project Lessons-Learned Document and agree:
Name

Project Title

Signature

Date

Bill Paulus

Business Sponsor

Email approval

6/16/09

Wayne Bowker

Technical Sponsor

Email approval

6/16/09

Vel Angamthu

Project Manager

The signatures above indicate an understanding of the purpose and content of this document by those
signing it. By signing this document, they agree to this as the formal Project Lessons-Learned Document.

LESSONS_LEARNED_REPORT BI Project

Page 3

Business Intelligence (BI) - Lessons Learned Survey Summary

1.

Are you satisfied with the finished deliverable?

Answer Options
Very
Somewhat
Not Very
Not at All
If not satisfied, what could have been done differently?

Response
Frequency

Response
Count

15.4%
38.5%
38.5%
7.7%

2
5
5
1
7

answered question
skipped question

13
0

Comments:
This is difficult to answer since I don't think the deliverable is finished.

I started out as a co-lead of a team but the other co-lead began heading in a different direction
working directly with the AVP.

Complete lack of up front planning. There no attempt to apply what was learned from the mistakes of
Crystal 10.

It is not finished. Specs and code were changed up until the last minute. There were no deliverable or
spec freezes so it seemed nothing got done. Also some key players were brought in at the final hour
with a different vision which caused tons of work to be discarded.

The project is nearing completion, but there is a lot of work ahead of us to operationalize this solution.

Because of the nature of the project, potential issues continued to surface up to the end of the
project. I am not sure what could have been done differently

An initial design (or requirements documents) should have been created before development started.

2.

How efficient and effective were project team meetings

Answer Options
Very
Somewhat
Not Very
Not at All
What would you change?

Response
Frequency

Response
Count

25.0%
50.0%
33.3%
0.0%

3
6
4
0
6

answered question
skipped question

12
1

Comments:
I really wasn't a part of these so I can't answer this one.

Most times not everyone was at meetings. Several times, no one showed. Not clear agendas or
deliverables

The project dropped from the sky. There was zero research of what they had and what they needed.

LESSONS_LEARNED_REPORT BI Project

Page 4

Each meeting seemed to be a recap of the last meeting. The org structure chart was handed out so
many times even when there were no new players in the room.

Project team meetings were not necessarily held regularly with one core team. I think the
effectiveness of the meetings varied by group.

Too many meetings at the beginning of the project and too few at the end. I felt I lost touch with the
project somewhat near the end.

3. Was the entire team committed to the project schedule?

Answer Options
Very
Somewhat
Not Very
Not at all
If not, what could have been done differently?

Response
Frequency

Response
Count

23.1%
46.2%
23.1%
7.7%

3
6
3
1
7

answered question
skipped question

13
0

Comments:

As much as we could be. This was a high priority but management did nothing to help us with the rest
of our workload.

See #2 above.

What project schedule?

I feel many competing projects caused people to not be as committed as they could have been.

I don't think that team members lacked interest or commitment, but there were competing priorities
that made it difficult to dedicate the time needed to meet all schedule deadlines.

I think the team was committed to the project schedule, but did not understand all the implications
and resulting time requirements until too close to the end of the project.

I thought we accomplished a lot near the end. More resource time for internal staff was available
than at the beginning.

LESSONS_LEARNED_REPORT BI Project

Page 5

4. How involved did you feel in project decisions?

Response
Frequency

Answer Options
30.8%
Very
38.5%
Somewhat
23.1%
Not Very
7.7%
Not at all
If you did not feel involved, what decisions did you feel left out of?

Response
Count
4
5
3
1
4

answered question
skipped question

13
0

Comments:

Decisions affecting data were made and not communicated to the entire group. Some of us knew
things and others didn't -- made it very difficult to test, make recommendations, etc.

It seemed like they wanted to put everything in BI, but didn't stop for a second to consider what was
already working and what was not.

The initial design.

Report Design. Ability to present options to Operations.

5.

How efficient and effective was communication between the project

sponsor, project manager and team members?

Answer Options
Very
Somewhat
Not Very
Not at all
What could have been done differently?

Response
Frequency

Response
Count

15.4%
53.8%
15.4%
15.4%

2
7
2
2
7

answered question
skipped question

13
0

Comments:

I did not hear much from the sponsors and when I did it was too late to do anything about it.

Vel did his best to keep everyone moving toward the same goal, but it feels like we haven't reached a
logical conclusion.

Goals and planning would have been a good start.

The project sponsor that represented my area did not communicate results from those meetings to
our organization.

Weekly status meetings and formal communications were very helpful.

Sometimes communication was done at a personal level, rather than a team level and this made it
difficult to ensure all team members were equally informed.

I didn't do very well at first but I put more emphasis on this towards the end. Needed to have more
input towards items that were assigned to me at the end of the project. Time for completion was
sometimes unreasonable and didn't felt I got enough advanced notice.

LESSONS_LEARNED_REPORT BI Project

Page 6

6.

How clearly defined were the objectives for this project?

Response
Frequency

Answer Options
16.7%
Very
41.7%
Somewhat
33.3%
Not Very
8.3%
Not at All
If not well defined, what could have been done differently?

Response
Count
2
5
4
1
5

answered question
skipped question

12
1

Comments:

I'm still not sure what is in and out of scope.

Never saw a single document.

At a high level, the business objectives and benefits for this project were clear and well documented.

This project was different than many technical projects because the objectives became more defined
as the project progressed. Requirements were defined incrementally which made the project more
difficult to manage

There did not seem to be a very clear scope for this project.

7. How clear were you on your role in the project?

Response
Frequency

Answer Options
41.7%
Very
25.0%
Somewhat
16.7%
Not Very
16.7%
Not at all
If your role was not clear, what could have been done differently?

Response
Count
5
3
2
2
3

answered question
skipped question

12
1

Comments:

Part of this is due to my manager not communicating this information.

They could have planned resource needs in advance.

I felt my role (or my perception of my role) changed throughout the project.

LESSONS_LEARNED_REPORT BI Project

Page 7

8.

How effective was the requirements identification process?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

0.0%
41.7%
41.7%
0.0%
16.7%

0
5
5
0
2
5

answered question
skipped question

12
1

Comments:

The fact that we have very different customers who need different things from a report was never
validated. We haven't completed this yet!

The desired infrastructure was clear, but specific reporting needs were developed throughout the
project by business owners.

Requirements sometimes seemed to be a moving target and some key decision makers were not
involved early enough in the project to ensure all requirements were clear. Once again, sometimes
requirements were identified incrementally, which is different from a more traditional project and that
makes it more difficult to assess the identification process.

Not enough of this was done up front.

Requirements were fairly well known for this phase of the project. Still due to the nature of the
project, new potential uses continued to come up which sometimes diverted attention.

LESSONS_LEARNED_REPORT BI Project

Page 8

9. How effective was the design (or implementation specifications)?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

25.0%
25.0%
16.7%
8.3%
25.0%

3
3
2
1
3
3

answered question
skipped question

12
1

Comments:

We were told BI is a design and build as you go along. But as new reports/needs were defined, then
the design changed and that impacted previously created reports. I felt like I was working on the
beach and every so often the tide would come in and wash my work away.

More University employee input may have been beneficial earlier in the project. Globus employees
were sometimes left to use their judgment instead of getting clear direction.

Not enough of this was done up front.

10. How effective were project team specification or design reviews?

Response
Frequency
Answer Options
16.7%
Very
16.7%
Somewhat
25.0%
Not Very
8.3%
Not at all
33.3%
I was not involved in this step
If not effective, what could have been done differently?

answered question
skipped question

Response
Count
2
2
3
1
4
3
12
1

Comments:

I did not see design reviews. I saw finished product and was asked to review it.

Changes were made without notifying the correct people. So people were using the universe or trying
to test data and were not getting the result expected (because they did not know a change was
made).

Liked the folder structure meeting(s).

LESSONS_LEARNED_REPORT BI Project

Page 9

11. How effective was the functional specifications?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

0.0%
33.3%
16.7%
8.3%
41.7%

0
4
2
1
5
3

answered question
skipped question

12
1

Comments:

Everything seemingly went into the Universe. I don't think many people will be able to navigate it. And
that will be worse as people move around. Where are the written specs?

Since this project did not follow a traditional waterfall approach, the iterative nature of the specdesign-test- implement makes it more difficult to assess the steps individually.

Not sure we really had any for the most part.

12. How effective was the test plan?

Response
Frequency
Answer Options
0.0%
Very
63.6%
Somewhat
9.1%
Not Very
0.0%
Not at all
27.3%
I was not involved in this step
If not effective, what could have been done differently?

answered question
skipped question

Response
Count
0
7
1
0
3
5
11
2

Comments:

Data validation continues to be a challenge.

Test plans were not clearly defined and were different depending on the type of data being analyzed.

Depends on the area. Not all areas had the same level of detail in their test plans.

Plan was ok but I only knew my part. Didn't have time to fully complete. Will continue to find issues
as the system is used.

LESSONS_LEARNED_REPORT BI Project

Page 10

13. How effective was the training plan?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

33.3%
16.7%
16.7%
8.3%
25.0%

4
2
2
1
3
6

answered question
skipped question

12
1

Comments:

Is there a training plan? There was much discussion about a need for one, but to date, I haven't seen
one.

A number of people went to two days of training. But what about the many people that will be
accessing the reports going forward?

Key members are trained, but there is a remaining activity to train and deploy for a wider FM audience
to be effective users.

The training plan was very clear, but the implementation steps were not completed exactly as defined
due to time constraints and staff turnover

Didn't really have a training plan.

I believe this is still being worked out?

14. How effective was the testing process?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

8.3%
50.0%
8.3%
8.3%
25.0%

1
6
1
1
3
4

answered question
skipped question

12
1

Comments:

It was effective, but too short and too much crammed into the last two weeks.

It was frustrating because you could test a report and be happy, then something would change some
where unrelated and your report change and you would have to re-validate it.

Detailed and defined testing should have started early in the project to provide adequate time for
changes and retesting

Depends on the area being tested. Not all areas were tested the same.

LESSONS_LEARNED_REPORT BI Project

Page 11

15. How effective was the interaction/cooperation between technical subteams?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not effective, what could have been done differently?

Response
Frequency

Response
Count

25.0%
41.7%
0.0%
8.3%
25.0%

3
5
0
1
3
4

answered question
skipped question

12
1

Comments:

Again, Vel tried but attendance was scarce.

Seemed everyone had their own universe and there was little cooperation between them. And at times
a change in one universe could negatively impact reports in another group. Changes to the universes
needed to be better communicated and agreed to before the change.

There is always a balance between the time invested by team members, and the benefit of increased
coordination.

At the beginning this was not very good. At the end of the project this was much better.

16. How effective was the deployment process?

Answer Options
Very
Somewhat
Not Very
Not at all
I was not involved in this step
If not efffective, what could have been done differently?

Response
Frequency

Response
Count

9.1%
45.5%
27.3%
0.0%
18.2%

1
5
3
0
2
5

answered question
skipped question

11
2

Comments:

It hasn't been deployed yet.

I think efforts to date have been effective but there is more to do.

There was not a traditional deployment process in this project

It hasn't really been officially deployed. It is currently in this half deployed half dev status.

System was deployed as it was developed.

LESSONS_LEARNED_REPORT BI Project

Page 12

17. For the next project, how or what could we improve on

in the way the project was conducted?
Response
Count

Answer Options

answered question
skipped question

8
5

Comments:

Management needs to do a better job of communicating to their staff and adjusting workloads to
accommodate the project.

Define the scope and budget before you start.

A code and spec freeze well before the final week of the project would be a good start. But there were
many decisions that were delayed early on that caused delays along the way...a software license,
defining what Tree to use, are two such examples.

More formal requirements gathering, project plan, and agreed upon deliverables at the onset would of
helped. As well as commitment by the entire team. I thought Vel and his team did the best they could
with what they we given. Even though there could of been things done better, we still accomplished a
lot and have a foundation for future growth, if we use it correctly.

For a future project of this type, I would like to see USIT take ownership of the application support
sooner in the project.

If it is possible, to more completely define the scope of this type of project early in the project that
should be done. In addition, input for executive decision makers should be received throughout the
project

Have a defined scope, functional specifications and design specifications. Also make sure the roles are
more clearly defined. Have better communication between all of the teams.

Do a better job defining requirements. Get sign off from more people at more levels of the
organization. Schedule the system down time more in the off-hours.

LESSONS_LEARNED_REPORT BI Project

Page 13