Volume 02 Issue 12-2022
97
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
ABSTRACT
In the current era of globalization, a new educational system is emerging around the world. the sole purpose of the
entire system is to improve the quality of education and digitize their evaluation system. This article reviews the
evaluation of the international TALIS program and key criteria for its improvement.
KEYWORDS
The Teaching and Learning International Survey (TALIS), principal survey, Survey Development, education system.
INTRODUCTION
This section describes features of the Teaching and
Learning
International
Survey
(TALIS)
2018
methodology including sample design and survey
design with a particular focus on the U.S.
implementation. For further details about the
assessment and any of the topics discussed here, see
the Organization for Economic Cooperation and
Development (OECD)’s TALIS 2018 Technical Report
(OECD 2019).
•
International Requirements for Sampling, Data
Collection, and Response Rates
•
Sampling in the United States
•
Survey Development
•
Translation and Adaptation
Research Article
TEACHING QUALITY MANAGEMENT BASED ON THE TALIS
INTERNATIONAL ASSESSMENT PROGRAM
Submission Date:
December 17, 2022,
Accepted Date:
December 22, 2022,
Published Date:
December 27, 2022
Crossref doi:
https://doi.org/10.37547/ajsshr/Volume02Issue12-15
Husanov Doston Rustamovich
Termiz State University Master's Students In The Specialty Of Management Of Educational Institutions
(Foreign), Uzbekistan
Journal
Website:
https://theusajournals.
com/index.php/ajsshr
Copyright:
Original
content from this work
may be used under the
terms of the creative
commons
attributes
4.0 licence.
Volume 02 Issue 12-2022
98
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
•
Recruitment, Survey Administration, and
Quality Assurance
•
Weighting
•
Data Limitations
•
Confidentiality and Disclosure Limitations
•
Statistical Procedures
•
International and U.S. Response Rates
•
U.S. Nonresponse Bias Analysis
•
References
International Requirements for Sampling, Data
Collection, and Response Rates
The OECD required all participating education systems
(countries and subnational regions) to adhere to the
TALIS 2018 technical standards, which provided
detailed information about the target population,
sampling,
response
rates,
translation,
survey
administration, and data submission. According to the
standards, the population covered in each education
system should consist of as many as possible of the
eligible International Standard Classification of
Education (ISCED) Level 2 teachers and school
administrators. Developed by the United Nations
Educational, Scientific and Cultural Organization
(UNESCO), ISCED is used by countries to map
education levels across countries and education
systems. In the United States, ISCED Level 2 teachers
are those of students in grades 7, 8, and 9 (also labeled
lower secondary education for convenience). To
provide valid estimates of teacher and principal
characteristics, the sample of TALIS teachers had to be
selected in a way that represented the full population
of ISCED 2 teachers in each education system. The
sample design for TALIS 2018 was a stratified
systematic sample, with the school sampling
probability proportional to the estimated number of
ISCED 2 teachers within each school. Samples were
drawn using a two-stage sampling process. In the first
stage a sample of schools was drawn, and in the
second stage a sample of teachers was drawn within
each selected school. Statistics Canada (StatCan), one
of the members of the TALIS consortium responsible
for the design and implementation of TALIS
internationally, drew the sample of schools for each
education system.
A minimum sample size of 4,000 teachers from a
minimum of 200 schools was required in each
participating education system. Following the TALIS
consortium guidelines, replacement schools were
identified at the same time the TALIS sample was
selected by designating as replacement schools the
two neighboring schools in the sampling frame. The
international guidelines specified that within schools, a
sample of 20 teachers was to be selected in an equal
probability sample unless fewer than 20 teachers were
available (in which case all teachers were selected).
Each education system collected its own data
following international guidelines and specifications.
The technical standards required that eligible teachers
Volume 02 Issue 12-2022
99
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
were those teaching at least one ISCED Level 2 class,
regardless of subject matter. School principals or head
administrators of each sampled school were also asked
to participate. School principal and teacher data were
collected independently, so teacher eligibility was not
dependent on principal participation (or vice versa).
The response-rate target was at least 75 percent of
schools and at least 75 percent of teachers across the
participating schools in each education system. A
minimum participation rate of 50 percent of schools
from the original school sample and 75 percent of
schools after replacement, was required for an
education system’s data to be included in the main
international comparisons. Education systems were
allowed to use replacement schools (selected during
the sampling process) to increase the response rate as
long as the 50 percent benchmark before replacement
had been reached.
The data collected by each participating education
system was adjudicated by the TALIS international
consortium to ensure that the data met the TALIS
technical standards for data collection. The principal
and teacher data were adjudicated separately. For
school-level data adjudication depended on only
school data (the principal participation) and for
teacher-level data, adjudication depended on only
teacher data (50 percent of teachers in the school had
to participate). Thus, for TALIS 2018, there are two
datasets
—
a school dataset with data from the
principal survey, and a teacher dataset with data for
teachers, each defined by their respective participation
standards.
Internationally, TALIS 2018 was designed to be as
inclusive as possible. In-scope teachers, those
comprising the international target population, were
all classroom teachers teaching at least one ISCED
Level 2 class and their school principals, and all subject
matters are included. Out-of-scope teachers included:
•
Teachers in schools for special education needs
students and their principals.
•
Substitute, emergency, or occasional teachers
who are defined as teachers filling in on a
temporary basis (no longer than 6 consecutive
weeks) for a teacher who is still employed as
either a full-time or part-time teacher at the
school.
•
Teachers teaching exclusively to adults who
are defined as teachers teaching only to adults,
whether the adult students follow a standard
or an adapted curriculum.
•
Teachers on long-term leave who are defined
as teachers “on long
-
term leave” who are
absent and not expected to be back during the
survey administration period.
•
Teacher aides who are typically non-
professional
or
paraprofessional
staff
supporting teachers in providing instruction to
students.
Volume 02 Issue 12-2022
100
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
•
Pedagogical support staff who provide
services to students to support the
instructional program, such as guidance
counselors or librarians.
•
Health and social support staff who are health
professionals such as doctors, nurses,
psychiatrists,
psychologists,
occupational
therapists, and social workers.
National target populations aim for maximum
coverage of eligible teachers and schools. However, in
some rare cases, for reasons of practicality, safety or
economy (e.g., remote schools, unique demographic
groups, types of schools, areas under civil unrest,
natural catastrophe), the national survey population
may be reduced. The U.S. population was not reduced
in any way. (See Table A-3 for coverage rates for each
country.)
Sampling in the United States
The TALIS 2018 school sample of 220 schools was
drawn for the United States by the TALIS consortium.
The U.S. sample was stratified into five explicit groups
based on school control (public/private) and grade
structure (with three groups
—
middle/junior high
schools with grades 6
–
8 or 7
–
9, high schools with
grades 9
–
12, or other schools with grade structures
that include at least one ISCED 2 grade). Given the small
number of private schools with a middle/junior high
school grade structure, this stratum was collapsed
with the high school grade structure in private schools.
Within each explicit stratum, schools were sorted by
census region (Northeast, Midwest, South, and West),
locale (urban/suburban/town-rural),1 percent minority
students,2 state, and number of ISCED 2 students
(measure of size
—
see paragraph below).
The school sampling frame included any school
containing at least one of grades 7 through 9 (defined
as ISCED 2 in the United States). The data for public
schools were from the 2015
–
16 Common Core of Data
(CCD) and the data for private schools were from the
2015
–
16 Private School Universe Survey (PSS). The U.S.
TALIS 2018 national school sample consisted of 220
schools. This number represents an increase from the
international minimum requirement of 200 and was
implemented
to
offset
anticipated
school
nonresponse and reduce design effects.
The CCD and PSS databases include estimates of the
number of teachers per school. While the number of
full-time equivalent teachers (FTE) was available from
the CCD, the number of teachers by ISCED level was
not. Following the advice of the TALIS consortium,
Westat, a research organization under contract to the
National Center for Education Statistics (NCES) in the
U.S. Department of Education, used the number of
estimated ISCED 2 students from the CCD and PSS
databases as the measure of size. Student estimates
rather than teachers were used due to the uncertainty
of the number of ISCED 2 teachers in certain types of
Volume 02 Issue 12-2022
101
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
schools (e.g., grade 9 teachers in grades 9
–
12 high
schools).
Overlap with the 2018 National Teacher and Principal
Survey (NTPS) school sample was minimized for the
2018 TALIS. The NTPS school frame was developed
from the 2014
–
15 CCD. The school sample for the NTPS
was selected by May 18, 2017, by the U.S. Census
Bureau, and the Census Bureau provided Westat with
the complete frame of schools (public schools only)
with NTPS probabilities of selection for every school on
their frame. Westat provided Statistics Canada the
NTPS probabilities for the schools that matched the
TALIS frame, and in drawing the U.S. TALIS sample
Statistics Canada used this information of minimize
overlap with NTPS.
In order to obtain a sample of teachers within schools,
participating schools provided a list of TALIS eligible
teachers (typically in January and February of 2018),
and with the data from this list the sample was drawn
using sampling software provided by the international
contractor.
Survey Development
The 2018 survey instruments were developed to cover
a set of predefined themes defined by the TALIS
framework (Ainley and Carstens 2018). These themes
were identified by representatives of the participating
education systems, the Questionnaire Expert Group
(QEG), and the OECD. Items were reviewed by the
TALIS QEG and by representatives of each country for
their relevance to TALIS’s
goals and for possible biases.
All participating education systems field-tested the
survey items in spring 2017. After the field trial, the
descriptive statistics and psychometric properties of
the items were reviewed, and items that did not meet
the established measurement criteria were dropped
for the main assessment.
The field trial also served as a means to test the field
operations for the TALIS survey. The field trial
evaluated the efficiency, accuracy, and effectiveness of
the online questionnaire system in capturing
information.
The final U.S. 2018 main study principal survey included
48 questions and the U.S. 2018 teacher survey included
55 questions. Countries were permitted to add
“national only” questions/answers and answer
categories. Also, each country adapted the
international questionnaire to fit national terms,
definitions, spelling, and punctuation, which is
explained further in the following section. The
resulting final set of items covered all 9 content
themes of the framework. The United States added
two questions to both the principal and teacher
surveys asking about the principal’s/teacher’s race and
ethnicity.
REFERENCES
Volume 02 Issue 12-2022
102
American Journal Of Social Sciences And Humanity Research
(ISSN
–
2771-2141)
VOLUME
02
I
SSUE
12
Pages:
97
-102
SJIF
I
MPACT
FACTOR
(2021:
5.
993
)
(2022:
6.
015
)
OCLC
–
1121105677
METADATA
IF
–
5.968
Publisher:
Oscar Publishing Services
Servi
1.
Ainley, J. and R. Carstens. (2018). Teaching and
Learning International Survey (TALIS) 2018
Conceptual Framework, OECD Education
Working Papers, No. 187, OECD Publishing,
Paris. https://doi.org/10.1787/799337c2-en .
2.
Judkins, D. R. (1990). Fay’s Method for Variance
Estimation. Journal of Official Statistics 6(3):
223-239.
3.
Kastberg, D., Cummings, L., Lemanski, N.,
Ferraro, D., Perkins, R.C., and Erberber, E.
(forthcoming). U.S. Technical Report and User
Guide for the 2018 Teaching and Learning
International Survey (TALIS). U.S. Department
of Education. Washington, DC: National Center
for Education Statistics.
4.
Krenzke, T., Roey, S., Dohrmann, S., Mohadjer,
L., Haung, W., Kaufman, S., and Seastrom, M.
(2006). Tactics for Reducing the Risk of
Disclosure Using the
NCES
DataSwap
Software. Proceedings of the Survey Research
Methods Section 2006. Alexandria, VA:
American Statistical Association.
5.
Organization for Economic Cooperation and
Development (OECD). (2019). TALIS 2018
Technical Report. Paris: Author.
