Data and Research

12.1. APPROACH TO REFORM DATA AND RESEARCH 12.2. THEORY OF CHANGE RESEARCH AND DATA 12.3. DATA 12.4. ADEQUACY OF DATA AVAILABILITY 12.5. STANDARDISED ANALYSIS 12.6. USE OF DATA 12.7. RESEARCH 12.8. PROGRAM MATRIX – DATA AND RESEARCH 12.9. Implementaion MATRIX 12.10. Result MATRIX

12. Data and Research
Education systems need to learn to improve. Data and research is a critical component that enables effective policy, planning and implementation. Education policies, products and processes target needs of children and, eventually, society. These needs are not static, and therefore the demand for data and research is continuous and calls for a culture of decision making that relies on empirical evidence support by a system that has the ability to produce well-researched analysis and quality data. There are major gaps in both areas. While there is some data availability and usage – though way below the required – research is completely absent. This despite mandate for research being legally available in most of the attached organisations of the Department of Secondary Education⁶⁰.
Resultantly, decisions are often taken in isolation of ground realities. The system has weak learning and products like curriculum and textbooks have a disconnect.

Approach to Reform Data and Research

Top

12.1. Approach to Reform Data and Research
There is a need to ensure that decisions and policies are backed by empirical evidence. This has two main requirements. Firstly, the system needs to have the capacity to produce need-based and reliable information. Secondly, demand for data and research for the development of policies, plans and routine implementation should be enhanced both from an individual approach perspective, as well as, institutional requirements. The overall goal of data and research is to ‘Ensure education policies, planning, monitoring and reform are based on empirical evidence derived from data and research.’

Theory of Change Research and Data

Top

12.2. Theory of Change Research and Data
Education policy, planning and implementation require ongoing information for improvements. Practically, there is a complete disconnect between research and data and resultantly education policies, plan and implementation are not not based on the ground realities that determine the needs of the child and society. There is a need to shift to a culture that believes in empiricism and practically builds evidence through research and data for decision making and execution.

---

60 These include Balochistan Textbook Board (BTBB), Bureau of Curriculum and Extension Centre (BOC&EC), Provincial Institute of Teacher Education (PITE), Policy, Planning and Implementation Unit (PPIU) and the Balochistan Board of Intermediate and Secondary Education (BBISE).

---

Figure 12-1 Theory of Change for Research and Data

 

PROBLEMS	SUB GOALS	PROGRAMS GOALS	SECTOR PLAN OBJECTIVE	GLOBAL, NATIONAL AND PROVINCIAL COMMITMENTS
There is a huge disconnect between realities of the child, needs of the society and the design and execution of education services due to absence of data, data analysis and research	Develop comprehensive needs based database for use in education policy, planning and service delivery	Ensure education policies, planning, monitoring and reform are based on empirical evidence derived from data and research	A culture of decision making that relies on empirical evidence and the ability of the system to produce well researched analysis and quality data	Global Commitments Sustainable Development Goals SDG4 -‘Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all’ SDG5- Achieve gender equality and empower all women and girls. SDG8- Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all. National Commitments Articles 25 A and 37b of the Constitution "The State shall provide free and compulsory education to all children of the age of five to sixteen years in such manner as may be determined by law.”   Provincial Commitments Balochistan Compulsory Education Act 2014 - providing a detailed legal framework for implementation of Article 25A
	Ensure standardised analysis is regularly conducted
	Institutionalise use of data in policy development, planning and service delivery
	Develop a culture and practice of research-based review of policies, plans and implementation processes

DATA

Top

12.3. DATA
Data production has improved since the last sector plan but still falls, majorly, short of needs. Data and its use is restricted to the Education Management Information System (EMIS). Balochistan has had a functioning EMIS since the early 1990s. It was restricted to the collection of data from government schools managed by SED on an annual basis and catered to limited needs of the Directorate of Education (Schools) only. In the last five years, based on the recommendations of the Balochistan Education Sector Plan, and the consequent capacity development plans, changes were made in the EMIS. Its scope and processes have been revised. EMIS has worked on data frameworks for other organisations: PITE, BOC&EC, BTBB and Directorate of Education. However, these have not rolled out. Despite the improvements in EMIS the province still lacks adequate and standardised data and has a weak culture of data use.

Adequacy of Data Availability

Top

12.4. Adequacy of Data Availability
Data available is inadequate. It limits the possibilities of analysis for policy development and implementation. Progress can barely be reported against major instruments like the Balochistan Compulsory Education Act 2014 and the SDG 4. Even the education sector analysis suffered through this lack of data. For example, an agreed, robust, data on out of school children is not available. The Plan uses the figures from Pakistan Education Statistics 2017 in the absence of a better source. Similarly, enrolments in private schools and madrassas are not available, though now being collected. The deficit is not limited to Balochistan. An important publication like the Pakistan Social and Living Measurement Survey has not been published since 2014-15. In some cases, data is available but not in a usable format because of non-linking with the central database or lack of digitisation.

Challenges

12.4.1. Challenges:
The following challenges were identified in the Education Sector Analysis with reference to Data Availability:
Absence of a comprehensive data needs analysis: A comprehensive need analysis for data has never been undertaken. Such an analysis is required at multiple levels. At the broadest level data requirements have to be driven by Balochistan Compulsory Education Act 2014 and the Sustainable Development Goals – especially- SDG4. To date, no analysis has been undertaken of the data needs for both. In the case of Balochistan Compulsory Education Act 2014, no attempt has been made to identify even indicators. In the case of SDG 4, no analysis has been made of the data needs against the indicators or to review the indicators framework to adjust to the priorities of Balochistan. Resultantly there is no reporting. The problem cascades down to micro-level data needs of individual organisations. The situation reflects the larger national problem of data. There are multiple sources representing data for sub-sets of issues without an overall need analysis. Systemic need analysis with an ongoing review is a requirement that will make data more targeted towards problems.
Absence of comprehensive data on schools other than SED: There are three types of schools other than those run by the Secondary Education Department. These include those run by government entities other than the Secondary Education Department, private schools and madrassas. Currently, a survey of private schools and madrassas has been initiated and initial results estimate about 40% of overall enrolment to be in private schools. Other public sector schools are not included. These are schools that are run by public sector organisations other than the Secondary Education Department (SED). The census at this point does not cover sufficient indicators (as compared to regular government schools). For better analysis, a more comprehensive data needs to be collected.
Limited digitisation of Data: There are disparate datasets that are available for digitisation in the department. Some are already digitised but not in the right format for public use. One example is the data of results of BBISE. It has both types of data. Historical one that is not digitised, and a more recent one that has been put into an electronic, but not in a user-friendly format. In fact, it is not even available publicly. Important data on textbooks produced and distributed is available but not digitised. There has been no effort made to locate other such data for digitisation and mainstreaming into the central database.
Unlinked disparate datasets: There is digitised data beyond the Education Management Information System (EMIS). This includes BAEC assessment results and the Non-formal Education Management Information System (NEMIS). Also, data on teacher training is being collected by PITE. These databases are not linked to the main Education Management Information System (EMIS). This means potential for critical information and analysis is not being availed.
Sub-goal is to ‘Develop comprehensive needs-based database for use in education policy, planning and service delivery.’

Strategies for Comprehensive Needs-Based Database

12.4.2. Strategies for Comprehensive Needs-Based Database:

1. Data need analysis will be conducted ensuring better planning, implementation and monitoring for enhanced quality, access and equity

A comprehensive need analysis will be undertaken to first identify the requirements and then align data collection and analysis with them. This will be undertaken through a comprehensive approach to look at data needs starting from macro-level indicators for Balochistan Compulsory Education 2014 and SDG 4 (along with other relevant SDGs), including, those on gender. It will be followed by an analysis of other more micro-level needs of each organisations.

2. Data collection on private schools, other public sectors and madrassas will be institutionalised

Without accurate data on children in schools a number of critical macro targets cannot be calculated, including, the actual number of out of school children. This will require a more comprehensive instrument for collection of data on private schools and madrassas, at par with the government schools. Resistance from the private school owners will be managed through negotiation and confidence building.

3. Existing data that is outside the main database of EMIS will be digitised

A lot of data that is available needs to be digitised. The most glaring omission are the results of the Boards of Intermediate and Secondary Education. Data exists in excel sheets and converted into more analytical formats used in EMIS. Also, older data available in hard copies can also be digitised. Similarly, a lot of information on personnel and finances should be digitised to link to the main EMIS.

4. Datasets will be linked for optimal information

The central EMIS will be linked up with other databases available within the system. Not only the existing ones but also others to be developed in future for stronger information availability and analysis.

Standardised Analysis

Top

12.5. Standardised Analysis
Dutiata analysis has improved but still not standardised. There have been some improvements from the past practices of simply publishing tables of data. EMIS now produces graphs and analysis with some level of disaggregation and detail. A lot of data available online can also be utilised for analysis in reasonably user-friendly processes. However, a structured, standardised and need based data analysis is required.

Standardised Analysis Challenges

12.5.1. Challenges:
The following challenges were identified in the Education Sector Analysis with reference to data analysis:
Absence of a standardised framework: No effort has been made to develop a structured need-based standardised framework for analysis. Partly because there has never been a proper data needs analysis that should have a standardised analysis as a product. This again creates issues of information availability, as per needs.
Capacity limitations: The two organisations that have the maximum need for data collection are the Policy Planning and Implementation Unit (PPIU) and the Education Management Information System (EMIS). While the actual need may come from a variety of sources the actual organisation into an analysis is the role of these two organisations. They do not have the capacity for any analysis beyond the rudimentary. This limits the possibilities of a need based framework for analysis and its production with data.
Sub-goal is to ‘Ensure standardised analysis is regularly conducted.’

Strategies for Standardised Analysis

12.5.2. Strategies for Standardised Analysis:

1. Standardised framework for analysis will be developed

Once data needs are determined through a structured approach, a standardised framework for data analysis will follow. This will allow more need-based reporting and the ability to compare progress in a structured manner. These frameworks will again flow from needs of the Compulsory Education Act 2014, SDG 4 and the Balochistan Education Sector Plan, among others.

2. Capacity of EMIS and PPIU to analyse data will be strengthened

As the main unit for planning the core ability will be developed within the Policy Planning and Implementation Unit (PPIU) and Education Management Information System (EMIS). This will require capacity building and close coordination between the two on an ongoing basis.

Use of data

Top

12.6. Use of data Finally, the most important and neglected area is the use of data. There is very limited use of systemic data. This results from a number of reasons.

Use of data Challenges

12.6.1. Challenges: There is no culture of data use. Importance of using data for decisions is not recognised in most government organisations including, to a great extent, in SED and its attached departments. Secondly, most officers do not have the capacity to use data effectively in decision making and finally, there is very limited space in the rules to mandate use of data. While this cannot be done for everything, there is a lot of scope for expansion of rules. The demand-side management is extremely critical as there is a correlation between the use of data and improvement in its availability and analysis.
Sub-goal is to ‘Institutionalise use of data in policy development, planning and service delivery.’

Strategies for the Use of Data

12.6.2. Strategies for the Use of Data:

1. Government officers and policy-makers will be sensitised on the importance of data use in decision making

Officers of the Department will be targeted through specific seminars, workshops and trainings to help them recognise the importance of data. This will be a regular annual feature.

2. Capacity for the use of data in policy, planning and service delivery will be developed

This capacity building will be more specific to the needs of each organisation to ensure that it is utilised effectively. This will also not be a one time activity but a regular feature for all new incumbents.

3. Existing rules will be reviewed and revised for mandatory data use in planning and decision making

Actions mandated by rules have a higher priority than those simply desirable. Unless rules are revised to make it compulsory to use data for decisions, most officers will not be compelled to do the same. However, not all decisions can be bound to the use of data. Current rules and procedures will be reviewed to see which of these can be linked to data use. This linking will be for different considerations, including, availability of data. Again, this does not necessarily have to be a one time review. With an increase in capacity of individuals, a shift in the culture of data use and greater availability of data, the scope will be further expanded in the future.

Research

Top

12.7. Research Research has been even more neglected than data. All organisations responsible for the production of quality products (BTBB, BBISE, BAEC, BOC&EC and PITE) along with PPIU have a research mandate. None of these have been operationalised due to lack of resources (both human and financial) which symptomises low priority according to research. Practically, except for some donor-funded initiatives, there is no systemic effort, or demand, for research and its link to policy development and review.

Research Challenges

12.7.1. Challenges:
The following challenges were identified in the ESA with reference to weak research:
Lack of recognition of the importance of research: Similar to data, the problem on the research side has also resulted from a lack of demand – a failure to recognise the importance of research in education. A clear indicator is the complete absence of funds for research functions of the organisations within SED. Secondly, even researches carried out in academia do not find any space among the policy-makers. Only with a change in attitudes will regular research be carried out within the department, and through academia.
Limited capacity for research: There is very little, if any capacity, within the government to carry out research or even collaborate with the academia on a research process. In fact, there have been no efforts to create linkages with academia. While the latter has more capacity for research, but due to years of disconnect with the public sector, they would require additional capacity building to connect with the needs of the Department, and the province as a whole.
No Rules to Mandate use of Research: As mentioned earlier in the chapter, there is a limited requirement of data use for certain plans. There is no such requirement for research. No policy, plan or decision is mandated to use research. While, again, similar to data (with greater limitations) use of research cannot be mandated for everything, but it can definitely be expanded to some basic level. Sub-goal is to ‘Develop a culture and practice of research-based review of policies, plans and implementation processes.’

Strategies for Research-Based Reviews

12.7.2. Strategies for Research-Based Reviews:

1. Awareness on the importance of research among senior decision-makers and political leadership will be enhanced

Similar to sensitisation on data specific, and ongoing, efforts for sensitisation of senior decision-makers and political leadership on research will need to be undertaken. The process will be conducted with the assistance of academia from both within and outside the province.

2. Regular need-based research will be conducted

There are many areas that require research. Besides, areas that require research will emerge. There will be an ongoing review of research needs and plans to conduct these researches through various sources, especially, the academia.

3. Capacity for research on education will be developed and strengthened

Capacity on research in education will be improved both within the department for organisations that have a direct mandate and also outside, including, academia. This will require a mutually supportive effort of the academia and the Secondary Education Department.

4. Research will be made a mandatory part of policy review processes and reforms

Actions mandated by rules have a higher priority than those simply desirable. Unless rules are revised to make it compulsory to use research for policy review processes, most officers will not be compelled to change their approach. However, not all policy review processes can be bound to the use of research. Current rules and procedures will have to be reviewed to see which of these can be linked to research. This linking can be for different considerations, including availability and ease of research. Again this does not necessarily have to be a one time review. With an increase in capacity of individuals, a shift in the culture of research and its use and greater availability of a research corpus the scope will be further expanded in future.

Program Matrix Data and Research

Top

12.8. Program Matrix – Data and Research

Table 12-1 Program Matrix – Data and Research

 

Program 6 – Data and Research
Overall Goal: Ensure education policies, planning, monitoring and reform are based on empirical evidence derived from data and research
Sub-Program	Sub-Goals	Strategies	Activities	Targets
6.1 Comprehensive Needs           Based Database	6.1.                      Develop comprehensive needs based database for use in education policy, planning and service delivery	6.1.1 Conduct data need analysis to ensure better planning, implementation and monitoring for enhanced quality, access and equity	I. PPIU to identify and document data needs for each organisation and for the overall education system, especially, the requirements of Balochistan Compulsory Education Act 2014, SDG 4 and 5 through engagement with each organisation.	Data     need     analysis conducted   Data           frameworks revised   Required data collected
			II. PPIU to coordinate with EMIS to review current data frameworks and revise them according to needs
			III. PPIU to identify other sources of data where EMIS data is insufficient of not needed
		6.1.2 Ensure that data on private schools, other public sectors and madrassas is comprehensively captured	I. Engage with private sector to agree on the data collection details and processes	Data on private schools and madrassas is regularly collected
			II. Engage with madrassas through the Home Department to agree on data collection details
			III. Collect data on private schools, madrassas and other public sector schools every two years
		6.1.3 Digitise existing data that is outside the main database of EMIS	I. Link existing digitised data to EMIS	All       existing       data digitised
			II. Identify data sets that can be digitised and made part of EMIS
			III. Digitise all such existing data into MIS’
		6.1.4 Ensure that datasets are linked for optimal information	I. Improve capacity of BTBB, DOE(S), PITE, BOC&EC and education secretariat to   roll   out,   manage   and   utilise    their	All data sets are rolled out and linked
			respective databases to be linked to the central EMIS.
			II. Roll out respective databases and link them
6.2     Standardised Analysis	6.2. Ensure standardised analysis is regularly conducted	6.2.1 Develop a standardised framework for analysis	I. Identify provincial requirements for data analysis in consultation with relevant organisations	Standardised framework develop   Data               collected according to the standardised framework
			II. Engage with the federal government to ensure alignment for provincial, national and international reporting needs.
			III. Develop a framework based on the analysis and collet data accordingly.
		6.2.2 Develop capacity of EMIS and PPIU to analyse data	I. Undertake capacity need assessment of EMIS and PPIU	EMIS       and       PPIU undertake regular data analyses
			II. Prepare and implement a capacity development plan
6.3 Use of Data	6.3. Institutionalise use of data in policy development, planning and service delivery	6.3.1 Sensitise officers of the department and policy makers on importance of data use in decision making	I. Develop a sensitisation campaign based on education data and its use	Sensitisation on use of data enhanced among officer          of          the Department             and education policy makers   Increased use of data in decision making
			II. Develop data sensitisation modules to be used in seminars and workshops
			III. Conduct workshops with officers and policy makers twice a year
		6.3.2 Develop capacity for use of data in policy, planning and service delivery	I. Undertake a training needs assessment for data use capacity in all organisations.	Capacity developed in policy, planning and service delivery
			II. Prepare a plan for training officers on use of data
		6.3.3 Review existing rules to ensure mandatory data use for planning and decision making.	I. Identify areas in planning, monitoring and implementation where data use should be essential	Rules mandating use of data notified
			II. Notify rules accordingly
6.4           Research Based Reviews	6.4. Develop a culture and practice of research-based review   of   policies, plans	6.4.1 Develop a plan to create awareness on the importance of    research    among   senior	I. Develop a plan for sensitisation of senior policy makers and political leadership through consultations and seminars	Increased use of research in decision making
	and            implementation processes.	decision makers and political leadership	II. Implement the plan
		6.4.2 Conduct regular need based research	I. Set up a committee with membership from all attachment departments to identify research needs for every 5 years with annual review	Enhanced research on education
			II. Create a research committee to engage with universities to conduct relevant research
			III. Set up a research fund to ensure research is pursued in areas identified
		6.4.3 Develop and enhance capacity for research on education	I. Operationalise and integrate the research mandates of BOC&EC, PITE, BTBB, BBISE & BAEC and identify capacity gaps as part of overall capacity development plans for these organisations	Capacity for research enhanced in the education system
			II. Recognise the importance of linkages between academia and government for research
			III. Mandate that have qualified academics to conduct research for the government and its students and teachers to be given recognition for their values and services
		6.4.4 Mandate research as part of policy review processes and reforms	I. Identify areas in planning, monitoring and implementation where research and its use should be essential	Rules mandating use of research notified
			II. Notify rules accordingly