Challenges and Recommendations for Data Collection Planning in TYPSS Implementation

Session 8/Report of an external mid-term review of Phase 1 of TYPSS implementation Break-out group discussions to workshop implementation/ prioritization of report recommendations Group 1 : Data Collections

Data collection 1. PLANNING 2. QUESTIONNAIRE DESIGN 3. MONITORING & SUPERVISION 4. ENTRY AND EDITING DISCUSS IN VIEW OF THE FOLLOWING: Challenges Risks Opportunities Priorities 2

Data collection group Members; Nauru Fiji Vanuatu Kiribati FSM Cook Is Samoa Solomon Is A.Samoa New Caledonia NZStat AusAid 3

Challenges Faced PLANNING NO NATIONAL MASTER PLAN A link to country national strategic development plans Plans for data collection exercises are not always reflected in the national plan Publication calendar plans showing dates of release of results (must be adhered to) BUDGETARY ISSUES: Securing funding in particular initial funding; internal and external funding support Need Government commitment Having to wait for annual budget cycles Donor funding conditions can cause practical operational issues 4

Challenges Faced PLANNING (cont) LACK OF COORDINATION BETWEEN DEPARTMENTS / INTERNATIONAL AGENCIES Various departments running surveys at same time Insufficient communication with NSO from international agencies running separate surveys (lack of recognition of NSO s role) LEGISLATION COULD BE IMPROVED NSO should have power to oversee all data collections in country NSO lacks the capacity to implement agreed legislation USER DEMANDS TOO LARGE AND UNCLEAR ESTABLISH STEERING COMMITTEE 5

Challenges Faced PLANNING (cont) UNREALSTIC TIME LINES BY USERS/DONORS IMPACT ON BUDGET OF DOMAIN REQUIREMENTS CHECK INFORMATION ON EXISTING ADMIN DATA SOURCES 6

Challenges Faced QUESTIONNAIRE DESIGN LACK OF CLEARLY DEFINED OUTPUTS/DEFITNITIONS Should take place prior to Q development HUGE RESPONDENT BURDEN Questionnaires getting too big Too much detailed requested from user which isn t practical LANGUAGE ISSUES Translation and back-translation ADAPT Q DESIGN TO TECHNOLOGY CONSISTENCY OF Q DESIGN BETWEEN SURVEYS 7

Challenges Faced FIELD SUPERVISION/MONITORING INSUFFICIENT MONITORING TOOLS AND PROCESSES DIFFICULT TO MONITOR REMOTE AREAS Have to rely on supervisors with little control by NSO Communication options are very limited at times TENDANCY FOR NSO STAFF TO WANT TO RELAX ONCE FIELD STAFF ARE DEPLOYED This is the time when NSO staff should be most active DIFFICULT TO ADDRESS CULTURAL ISSUES IN THE FIELD HARD TO ATTRACT THE RIGHT FIELD STAFF 8

Challenges Faced FIELD SUPERVISION/MONITORING TRANSPORT AVAILABILITY PUBLICITY/ADVOCATION 9

Challenges Faced DATA ENTRY AND EDITING SYSTEM NOT IN PLACE IN TIME POOR QUALITY OF STAFF UNDERTAKING DATA ENTRY COMPLEX SURVEYS = COMPLEX SYSTEMS DATA SETS AREN T PROPERLY FINALISED BY THE TIME OF DATA ANALYSIS IMPLEMENTATION OF NEW TOOLS EDITS SPECS NOT CLEARLY DEFINED 10

Risks PLANNING SURVEY NOT COMPLETED ON TIME SURVEY GOES OVER BUDGET DELAY IN FUNDING SURVEY WON T MEET ALL DESIRABLE OBJECTIVES 11

Risks QUESTIONNAIRE DESIGN KEY SURVEY OUTPUTS AREN T PRODUCED KEY INDICATORS ARE NOT DERIVED CORRECTLY USERS NOT SUFFICIENTLY INVOLVED IN Q DESIGN DATA GAP IN CONTENT IF SAME Q S NOT USED OVER TIME DATA UNDER-UTILISED 12

Risks FIELD MONITORING / SUPERVISION FIELD STAFF DO AS THEY PLEASE Can even runaway Fill in forms under coconut tree SUPERVISERS DON T SUPERVISE INFORMATION COMES BACK BAD Forms missing Forms not filled in Many errors in forms STAFF TURNOVER TOO MANY CALLBACKS 13

Risks DATA ENTRY / EDITING MANY DATA ENTRY ERRORS INSUFFICIENT DATA EDITING Data won t be as clean as desirable CAN DELAY THE ANALYSIS STAGE OF THE SURVEY 14

Opportunities PLANNING SECURE LONG TERM DONOR FUNDING WELL DEVELOPED MoUs AT THE START OF EACH SURVEY WITH KEY PARTNERS Include all TA aspects of survey INNOVATION IN NEW TOOLS IMPROVE ADMIN DATA REDUCE WORKLOAD OF SURVEYS DEVELOPMENT OF NSDS FOR ALL PICTS STANDARDISATION OF METHODOLOGY ADOPTION OF REGIONAL/INTERNATIONAL STRATEGIES ON OUTPUT PRODUCTION METADATA DEVELOPMENT (SYSTEM) 15

Opportunities QUESTIONNAIRE DESIGN STANDARISED QUESTIONNAIRES FOR KEY SURVEYS (CENSUS, HIES, DHS) MORE SOUTH-SOUTH COOPERATION OPPORTUNITES TA ASSISTANCE IN GENERAL MADE EASIER BY STANDARDISED Q COMPARABILITY BETWEEN PICTS MADE POSSIBLE ADOPTION OF INTERNATIONAL/REGIONAL STANDARDS AND CLASSIFICATIONS WHERE APPLICABLE TEST QUESTIONNAIRES 16

Opportunities FIELD SUPERVISION AND MONITORING CAPACITY BUILDING OF NSO STAFF More accountable CULTIVATE GOOD PRACTICES AND ETHICS OF NSO STAFF FOR FIELD WORK SUPERVISION FOR FUTURE SURVEYS POOL OF SKILLED FIELD STAFF NEW TECHNOLOGIES FOR FIELD STAFF COMMUNICATION 17

Opportunities DATA ENTRY / EDITING DEVELOPING STANDARDISED SYSTEMS ENABLES ADOPTION IN OTHER COUNTRIES SYSTEM CAN ALWAYS BE IMPROVED OVER TIME ADOPTION OF NEW TECHNOLOGIES EG, PDA 18

Opportunities DATA ENTRY / EDITING DEVELOPING STANDARDISED SYSTEMS ENABLES ADOPTION IN OTHER COUNTRIES SYSTEM CAN ALWAYS BE IMPROVED OVER TIME ADOPTION OF NEW TECHNOLOGIES EG, PDA CLASSIFICATION MANAGEMENT SYSTEM 19

Priorities PLANNING FUNDING AVAILABILITY GOVERNMENT COMMITMENT CLEAR DEFINITION OF OUTPUTS SKILL STAFFING AVAILABILITY STICK TO SURVEY TIMETABLE INVOLVEMENT OF USERS 20

Priorities QUESTIONNAIRE DESIGN AGREED OUTPUTS WITH STAKEHOLDERS/USERS QUESTIONNAIRE STANDARDIZATION WHERE POSSIBLE ADOPTION OF REGIONAL/INTERNATIONAL STANDARD CLASSIFICATION TESTING OF Q S 21

Priorities FIELD SUPERVISION/MONITORING REQUIRED SUPERVISION/MONITORING TOOLS NEED TO BE IN PLACE SKILLED AND COMMITTED FIELD OFFICERS GOOD RECRUITMENT GOOD TRAINING STRONG LEADERSHIP/MANAGEMENT FROM NSO SOUTH-SOUTH COLLABORATION WHERE POSSIBLE 22

Priorities DATA ENTRY/EDITING WELL TRAINED STAFF CAPACITY BUILDING STANDARDIZATION WHERE POSSIBLE PROCEDURES WELL DOCUMENTED QUALITY ASSURANCES IN PLACE TIMELY DEVELOPMENT OF SYSTEM NEW TOOLS INTRODUCED WHEN APPLICABLE 23