VTQ Standards Regulatory Approach and Team Management Insights

VTQ standards regulatory approach Ofqual Conference Bryan Horne Associate Director, VTQ Standards 28 February 2017

Content VTQ Standards team Portfolio management approach Risk rationales Thematic work planned for 2017 - expected outcomes Our approach to thematic work: malpractice as an example Incident and event handling Statement of Compliance 2017 Close and questions

VTQ Standards team Bryan Horne Associate Director Caroline Richards Senior Manager Richard Knee Senior Manager Senior Manager - vacancy Sue Budworth Senior Manager Senior Manager vacancy Executive Officer - vacancy Michael Larkin Senior Officer Sofiya Saleh Senior Officer Sarah Gibbons Senior Officer

Portfolio management VTQ Standards staff manage a portfolio of ~25 awarding organisations each. Oversee awarding organisations management of event notifications. Respond to queries. Work with audit team to target awarding organisations for audit activity. Act as a focal point within Ofqual in terms of knowledge of an awarding organisation s operations. GQ Standards team lead on examination boards such as OCR and Pearson Education Ltd, but VTQ Standards support where there are VTQ related matters.

Risk rationales VTQ Standards uses a range of intelligence to develop risk rationales which drive its programme of supervisory work. This typically includes, but is not limited to: information from past audits and technical evaluations of qualifications, complaints to Ofqual, event notifications, stakeholder intelligence, data sources scale and scope of certifications, and recognition applications.

What event notifications have told us

What data analysis might typically tell us Numbers of Construction qualifications offered - Q1, 2016 by AO Construction certifications - Q1, 2016 by AO WJEC-CBAC Training Qualifications UK Ltd Highfield Awarding Body for Compliance Open University Awarding Body Council for the Curriculum, Examinations and OCN North East Region NCFE MP Awards WJEC-EDEXCEL-CBAC Open College Network West Midlands Defence Awarding Organisation Agored Cymru Constructing Excellence in Learning Pearson EDI Agored Cymru National Federation of Property Professionals Pearson EDI Defence Awarding Organisation Open Awards OCR Industry Qualifications AQA - City & Guilds Chartered Institute of Building Open Awards NCFE Constructing Excellence in Learning Limited AIM Awards BPEC Certification Ltd Highfield Awarding Body for Compliance Royal Society for Public Health Logic Certification Limited Logic Certification Limited GQA Qualifications Limited ABC Awards Open College Network Eastern Region NOCN NOCN GQA Qualifications Limited Excellence, Achievement & Learning Excellence, Achievement & Learning Limited British Safety Council Pearson Education Ltd City and Guilds of London Institute Cskills Awards 0 100 200 300 400 500 600 0 5000 10000 15000 20000 City and Guilds of London Institute offered 355 qualifications and issued 17,780 certificates in that quarter (~ 50 certificates per qual. on average), whereas Cskills Awards who offered 544 qualifications issued 9,965 certificates (~18 certificates per qual. on average).

What last years audits told us A range of findings, helping to inform our approach in 2017: Construction and the Built Environment not the same issues as the security sector, but outputs informing our approach to malpractice and quality assurance controls, published our regulatory report about the security sector in January ~ centre control and malpractice risks, and Applied Generals - looking at delivery of qualifications in the Department for Education (DfE) performance tables ~ we will continue with a programme of work in this area.

Leading to an annual programme of supervisory work Work is being planned so that we minimise regulatory burden. We will consider regulatory impact throughout our work. Phasing to avoid busy periods such as the summer examination series. Wherever possible work is being co-ordinated, thereby eliminating duplication. If we have checked an area already and evidence is still current, we will not ask for it again. Work now underway * This is an indicative table for illustrative purposes only. If selected for aspects of our work, you will be informed in advance.

Thematic work - 1 Malpractice seeking assurance that malpractice is being picked up, tackled appropriately, and necessary improvements made to deter/prevent. Quality assurance controls assurance that these are in place and operating effectively, particularly in the following areas: where centres carry out internal assessment, and correct and timely certification focussing on IT usage and arrangements. Specification of Apprenticeship Standards for England (SASE) assurance that appropriate controls are in place where qualifications are within SASE frameworks that are either being wound down , or delivered at pace.

Thematic work - 2 Follow-up activity where specific risks have been identified through audits (2016) security sector checks, assurance that these have been addressed. Qualifications that are/have been submitted to DfE for its school/college performance tables seeking assurance through technical checks that particularly where DfE has concerns, qualifications meet our requirements. Applied Generals seeking assurance that standards for these qualifications are being set and maintained appropriately. Other baseline/reactive checks where we require assurance.

Our approach to thematic work Risk assessed Approach to check, mitigate developed Impact of work measured Checking, mitigation work carried out Measures of effect identified

Our approach to thematic work - malpractice Stage Malpractice Unsafe awards standards, public confidence affected Risks to objectives Scrutiny of incidents, checking resources & arrangements to tackle, research into risks presented, AO/stakeholder symposium, additional guidance? Approach to check, mitigate Reduced amounts of malpractice (real), accurate reporting and management where it does occur, greater assurance levels through audits, increased public confidence Measures of effect Scrutiny of incidents, audit programme checking controls Checking Impact measured Tracking reporting trends and patterns over time, capturing levels of assurance through audits, user surveys

Incident and event handling Event reported to Ofqual We might ask you about root causes Questions to find out scale/scope? You might have told us that you were non-compliant. We might make a determination of non-compliance AO manages event Further dialogue if necessary, intervention? We might seek further assurance We might close off from our perspective with no further action AO reports closure During event management After event management

Statement of Compliance - 2017 Window for submissions as 2016 (June-September). Process as 2016. We will publish guidance as 2016.

Close Continue to submit event notifications as the Conditions require. We will be transparent in our approach tell you about work beforehand, tell you what we ve found, publish where appropriate. We are a risk based regulator and we will continue with this approach through 2017.

Questions