The 5Ws and H of

the Big V
A webinar designed to help basic understanding of validation

- Michelle Charlton, July 2017

Write On Resources
RTO Resources and Training Products Australia PD in VET
Write On Resources Version 2.1, May 2017
About the Presenter
Michelle first entered workplace training in 1993. Since then, she has
worked for a variety of organisations of various sizes, in both the public
and private sector. Michelle has a passion for learning and development
and looks to cultivate and share this at every opportunity.
Her work experience and vocational qualifications include those in
workshop facilitation and presentation, leadership and management,
project management, and business operations.
In recent years, Michelle has worked within the VET sector as a trainer
and assessor, and RTO branch manager, before focusing on Instructional
Design.
Michelle has tertiary qualifications in psychology, adult and vocational michelle@writeonresources.com.au
education, and a Master of Training and Development. writeonresources.com.au
She currently works as a resource developer and industry consultant, and
some of her contracts include those with peak industry associations, Write On Resources
public and private RTOs. @writeonresources
Control Panel
Audio test

Hands up who can hear me

Raise your hand to

indicate
Whos online today?

POLL
Aim for Todays Session
To confirm understanding of validation
To confirm understanding of legal requirements regarding validation
Explore the 5Ws and H of the Big V
What is
Validation?
V is for .
Take a moment to write in
the chat window your
understanding of what
validation means
The Many Faces of Validation
Checking assessment tools are designed to gather appropriate evidence
Confirming judgements of competence are aligned across assessors
Making sure current industry practices are reflected in assessments
Checking that all unit requirements are met
The act of checking or proving the accuracy/acceptability of something
Part of continuous improvement in a Quality System
The Many Faces of Validation YES
YES

Checking assessment tools are designed to gather appropriate evidence

Confirming judgements of competence are aligned across assessors
YES
Making sure current industry practices are reflected in assessments
Checking that all unit requirements are met YES

The act of checking or proving the accuracy/acceptability of something

Part of continuous improvement in a Quality System
YES

YES
So many interpretations all correct
How do we know which one to use?

How do we know how to do what is expected of us?

How do we make sure we are doing what we are supposed to do?

The Official
Line
V is for validation is for
Definition from Australian Legislative Instrument, Standards for
Registered Training Organisations 2015:
Validation
is the quality review of the assessment process. Validation involves checking that
the assessment tool/s produce/s valid, reliable, sufficient, current and authentic
evidence to enable reasonable judgements to be made as to whether the
requirements of the training package or VET accredited courses are met. It
includes reviewing a statistically valid sample of the assessments and making
recommendations for future improvements to the assessment tool, process
and/or outcomes and acting upon such recommendations.
(Users Guide to the Standards 2015, Glossary)
So what does that mean?
Validation is:
The quality review of the assessment process
Checking assessment tools are well designed must be able to produce required
evidence: valid, reliable, sufficient, current, authentic
Checking reasonable judgements of competency can be made
Competency = requirements of Training Package/VET accredited course are met
Reviewing assessments checking reasonable judgements of competency have
been made
Reviewing assessments to make suggestions for continuous improvement to the
tool and/or outcomes
Using a statistically valid sample upon which to base the review
(making a plan to) Act upon recommendations that come from the review
Lets park this slide for a moment
V is for validation is for
Outlined by Australian Legislative Instrument, Standards for Registered
Training Organisations 2015:
Validation
Is addressed in the requirements of Clauses 1.9 1.11

RTOs regulated by ASQA must comply with these requirements

Plenty of elements

How do we tie them all together?

5Ws and H of
the Big V
The Big V
Definitions and outlines are one thing. Legislative requirements are
another.
Clauses for action are yet another.

Is it any wonder, that for some people, validation becomes an

implementational minefield?

Good thing we are here today

Clause 1.9 The Plan
The RTO implements a plan for ongoing systematic validation of
assessment practices and judgements that includes for each training
product on the RTOs scope of registration:
when assessment validation will occur;
which training products will be the focus of the validation;
who will lead and participate in validation activities; and
how the outcomes of these activities will be documented and acted
upon.
The Plan And Planning For a Plan
**Before a plan can be
implemented, it must be
designed!

Luckily we have some direction

on which elements must be
included in our plan design
Clause 1.10 The Deadline
For the purposes of Clause 1.9, each training product is validated at
least once every five years, with at least 50% of products validated
within the first three years of each five year cycle, taking into account
the relative risks of all of the training products on the RTOs scope of
registration, including those risks identified by the VET regulator.
 We are
here
The Deadline Start Deadline 1 Deadline 2
1 April 2015 1 April 2018 1 April 2020

Timeline for Standards validation cycle

The first three years of the first

five year cycle is almost up

1 April 2018
Clause 1.11 The People
For the purposes of Clause 1.9, systematic validation of an RTOs assessment
practices and judgements is undertaken by one or more persons who are not
directly involved in the particular instance of delivery and assessment of the
training product being validated, and who collectively have:
a) vocational competencies and current industry skills relevant to the
assessment being validated;
b) current knowledge and skills in vocational teaching and learning; and
c) the training and assessment credential specified in Item 1, or
Item 2, or Item 4, or Item 5 of Schedule 1. BUT! On or after 1 April
2019
Industry experts may be involved in validation to ensure there is the the training and
combination of expertise set out in (a) to (c) above. assessment credential
specified in
Item 2 or Item 5 of
Schedule 1.
The People
Specific instructions about who can be involved
and what qualifies them to be involved
Special Note RE: The People
Standards for RTOs Amendment 2017
PRE-1 April 2019
c) the training and assessment credential specified in
Item 1, or Item 2, or Item 4, or Item 5 of Schedule 1.

POST-1 April 2019

c) the training and assessment credential specified in
Item 2, or Item 5 of Schedule 1.
 PRE-1 April 2019
Schedule 1
ITEM # DESCRIPTION
1 TAE40110 Certificate IV in Training and Assessment
2 TAE40116 Certificate IV in Training and Assessment or its successor
OR
TAE40110 + TAELLN411/TAELLN401A
PLUS
TAEASS502 Design and develop assessment tools or its successor or TAEASS502A or TAEASS502B
4 TAESS00001 Assessor Skill Set
5 TAESS00011 Assessor Skill Set or its successor
OR
TAESS00001
PLUS
TAEASS502 Design and develop assessment tools or its successor or TAEASS502A or TAEASS502B
 POST-1 April 2019
Schedule 1
ITEM # DESCRIPTION
2 TAE40116 Certificate IV in Training and Assessment or its successor
OR
TAE40110 + TAELLN411/TAELLN401A
PLUS
TAEASS502 Design and develop assessment tools or its successor or TAEASS502A or TAEASS502B
5 TAESS00011 Assessor Skill Set or its successor
OR
TAESS00001
PLUS
TAEASS502 Design and develop assessment tools or its successor or TAEASS502A or TAEASS502B
And yet any the wiser?
3 clauses, but LOTS
of information:
To dissect,
Understand,
Plan for,
Tie together, and
Implement
First things first
Before we can tie it all together, we need to break it down
5Ws and H
To help us understand what is required for validation activity according
to the Standards, use:
Who
What
When
Where Which
Why
How
The definition gives us areas of focus
Validation is:
The quality review of the assessment process
Checking assessment tools are well designed must be able to produce required
evidence: valid, reliable, sufficient, current, authentic
Checking reasonable judgements of competency can be made
Competency = requirements of Training Package/VET accredited course are met
Reviewing assessments checking reasonable judgements of competency have
been made
Reviewing assessments to make suggestions for continuous improvement to the
tool and/or outcomes Remember that slide we
Using a statistically valid sample upon which to base the review parked?
Time to let the brake off

(making a plan to) Act upon recommendations that come from the review
According to the definition, Validation
focuses on:
Assessment processes WHAT
Design of assessment tools and what they can produce
Required evidence: valid, reliable, sufficient, current, authentic it isabout
Whether reasonable judgements of competency can be made
Whether requirements of Training Package/VET accredited course are met
What judgements of competency have been made
Using a statistically valid sample
Suggestions for continuous improvement to the tool and/or outcomes
Making an action plan
According to the definition, Validation
requires:
Quality review
Checking WHAT
Of the design evidence required
Reasonable judgements can be made it involves
Unit/Accredited Course requirements are met
Reviewing
Assessment judgements made
Tools to make suggestions for improvement
A statistically valid sample
Planning
Forming a plan for improvement
Acting on the plan for improvement
Ok, but
HOW is generally the biggest unknown

What am I supposed to do?

Lets see what the Standards say
Clause 1.9 - The PLAN
The RTO implements a plan for ongoing systematic validation of
assessment practices and judgements that includes for each training
product on the RTOs scope of registration:
when assessment validation will occur;
which training products will be the focus of the validation;
who will lead and participate in validation activities; and
how the outcomes of these activities will be documented and acted
upon.
Clause 1.9 The Plan Overarching
WHAT
A plan for:
Ongoing systematic validation of assessment practices and judgements
Including each training product on the RTOs scope of registration
WHAT to include

Training Product
means AQF qualification, skill set, unit of competency, accredited short
course and module (Users Guide to the Standards 2015, Glossary)
Clause 1.10 The Deadline WHEN
each training product is validated at least once every five years,
HOW many
with at least 50% of products validated within the first three years of
each five year cycle,

WHAT else must be considered

taking into account the relative risks of all of the training products on the
RTOs scope of registration, including those risks identified by the VET
regulator.
Clause 1.11 The People WHO
systematic validation of an RTOs assessment practices and
judgements is:
HOW many
Undertaken by one or more persons
WHAT involvement
Who are not directly involved in the particular instance of delivery and
assessment of the training product being validated*,

*(ASQA have conceded that for smaller RTOs, people power isnt always accommodating, so a trainer
who was involved in the particular assessment event may be involved, BUT they cannot be the lead
validator)
 Clause 1.11 The People: WHO cont
[Validation is] undertaken by one or more persons who collectively have:
vocational competencies and current industry skills relevant to the
assessment being validated; WHAT experience
current knowledge and skills in vocational teaching and learning; and
WHICH qualifications
the training and assessment qualification or assessor skill set referred
to in Item 1, or Item 2, or Item 4, or Item 5 of Schedule 1.
Industry experts may be involved in validation to ensure there is the
combination of expertise set out in (a) to (c) above.
WHO else
 Making a Plan Come Together
4Ws and H Explanation Plan Detail
When Validation will occur Put in your selected date(s)/time(s)
When
> Which > Training products (TPs) will be the Nominate the training products to be validated at each time
focus at each scheduled time? Check the ASQA site for risk updates
> Consider relative risk of each TP
Who Will lead? Name the roles/people to lead
Will participate? Name the roles/people to be involved including any industry
(Altogether how many?) SMEs
Cannot be involved? Remember the stipulations in Clause 1.11
Who
> What > Experience and/or qualifications for List the qualifications required to validate the TPs in focus
each person Cross reference all necessary quals/experience are collectively
met by the validator or validation group
How Many assessments must be included? Calculate the statistically valid sample size
How Will outcomes be recorded? Nominate your plan for making the next plan
Will outcomes be actioned? Nominate your plan for implementing the next plan
The Fifth W . WHY
Validation is a compliance requirement the Standards say so
Validation is a best practice activity your $$ say so
Validation is about QUALITY outcomes for your students say so
Remember too, WHAT it is about
THE PLAN
4Ws and H Plan Detail

When Put in your selected date(s)/time(s)

When
> Nominate the training products to be validated at each time
Which Check the ASQA site for risk updates

Who Name the roles/people to lead

Name the roles/people to be involved including any industry SMEs
Remember the stipulations in Clause 1.11

Who
> What List the qualifications required to validate the TPs in focus
Cross reference all necessary quals/experience are collectively met by the
validator or validation group

How Calculate the statistically valid sample size

How Nominate your plan for making the next plan

Nominate your plan for implementing the next plan
Remember too, WHAT it involves
THE PLAN
4Ws and H Plan Detail

When Put in your selected date(s)/time(s)

When
> Nominate the training products to be validated at each time
Which Check the ASQA site for risk updates

Who Name the roles/people to lead

Name the roles/people to be involved including any industry SMEs
Remember the stipulations in Clause 1.11

Who
> What List the qualifications required to validate the TPs in focus
Cross reference all necessary quals/experience are collectively met by the
validator or validation group

How Calculate the statistically valid sample size

How Nominate your plan for making the next plan

Nominate your plan for implementing the next plan
Wait a second!
Multiple time points!
Future tense:
Whether the tools are
well designed to meet
all requirements

Past tense:
Outcomes of having
used the tools
A Two Part Plan
Simply means our plan must address validation at two time points

At a minimum, assessment tools should be validated:

BEFORE use to answer the basic Q can they do the job?
and then again, AFTER use to answer the basic Q did they do the
job?

However, this is not a case where less is more

An holistic approach
Validation of an
assessment system

Before
During
After
Before
During
After
Its worth mentioning old friends
Validation v Moderation
Past definitions, interchangeable and confusing!
Main difference is the timing
Moderation is balancing judgments made so
after assessment is completed by the student
and after a competency judgement awarded by
the assessor. But! Before the assessment
decision is communicated to the student
Validation is a quality review process so,
before, during and after, all apply
Questions?
About the Presenter
Michelle first entered workplace training in 1993. Since then, she has
worked for a variety of organisations of various sizes, in both the public
and private sector. Michelle has a passion for learning and development
and looks to cultivate and share this at every opportunity.
Her work experience and vocational qualifications include those in
workshop facilitation and presentation, leadership and management,
project management, and business operations.
In recent years, Michelle has worked within the VET sector as a trainer
and assessor, and RTO branch manager, before focusing on Instructional
Design.
Michelle has tertiary qualifications in psychology, adult and vocational michelle@writeonresources.com.au
education, and a Master of Training and Development. writeonresources.com.au
She currently works as a resource developer and industry consultant, and
some of her contracts include those with peak industry associations, Write On Resources
public and private RTOs. @writeonresources