BS ISO/IEC 23988:2007

Foreword
Introduction
1 Scope
2 Normative references
3 Terms and definitions
4 Guiding principles
5 Interface between assessment content and IT delivery
6 IT delivery of assessments - general
7 Software for IT delivery of assessments - navigation and
   usability issues
8 Instant automated scoring, result determination and feedback
9 Preparation and transmission of assessment content and correct
   responses
10 Software and procedures for recording and transmission of
   candidates' details and responses and for their storage by
   assessment distributors
11 Provision of instructions and assessment-specific information
   for assessment centres
12 Provision of information and practice material for candidates
13 Equipment and facilities at assessment centre, including storage
   of responses
14 Staffing of assessment centre
15 Preparation for the assessment session
16 Conduct of assessment session
17 Procedures relating to emergencies, technical failures and
   irregularities
Annex A (informative) Scenarios
Bibliography

Provides a means of showing that the delivery and scoring of the assessment are fair and do not disadvantage some groups of candidates, for example those who are not IT literate.

1.1 General This Standard gives recommendations on the use of information technology (IT) to deliver assessments to candidates and to record and score their responses. Its scope is defined in terms of three dimensions: the types of assessment to which it applies, the stages of the assessment “life cycle” to which it applies and this Standard’s focus on specifically IT aspects. 1.2 Types of assessment This Standard is relevant to a wide range of assessments, including: assessments used in education (both compulsory and post-compulsory), training and compliance (e.g. compliance with legislation relating to health and safety or financial services); assessments of knowledge, understanding and skills (i.e. “achievement tests”), but not psychological tests of aptitude and personality; high-stakes assessments and examinations and also low-stakes assessments used for feedback on progress, identification of learning needs, self-assessment and remediation; assessments which include feedback, as well as those which provide only a result; both fixed-date, test-windows and on-demand assessments; both items which can be scored by computer and the delivery (but not scoring) of items or tasks which are delivered using IT, but have to be referred to a human marker for scoring; however, the use of IT for scoring lengthy free-text responses (“essays” and similar) is excluded; a wide range of computer-scorable assessments, including not only “objective tests” (multiple-choice and other item types), but also assessments of keyboarding skills and software use; NOTE Items can include graphics, multimedia and access to data, case studies, etc. This Standard does not, however, cover the use of specialist simulations (e.g. of industrial processes or driving/piloting). assessments taken in universities, academies, colleges, high-schools, training centres, schools and assessment centres and also assessments taken in less formal settings, including learning centres and in the workplace; assessments which are set, taken and scored within a single organisation (for examplea university or a company) and assessments taken in an organisation separate from the one responsible for the assessment content; assessments delivered in a variety of ways, including on-line, on local networks and on stand-alone computers. Clause 9 of this Standard is also applicable to the use of IT for the transmission of candidates’ coursework. 1.3 Assessment life cycle Although assessment procedures vary, the typical life cycle of assessment consists of the following steps: identification of need to assess; design of outcomes/assessment methodology; preparation and calibration; pre-registration (includes payment); distribution1); authentication (includes identification)1); delivery1); response return1); scoring, result determination and/or feedback1); data return1); analysis; appeals; certification. Preparation of assessment content is outside the scope of this Standard, but where pretesting is undertaken using IT for delivery, the relevant clauses of this Standard should be taken into consideration. 1.4 Focus on specifically IT aspects The emphasis throughout this Standard is on the additional or different measures which should be taken as a result of the introduction of IT. Measures which are common to both paper and IT-delivered systems are either omitted or covered in only general terms. This Standard does not cover the purely pedagogical aspects of assessment preparation or scoring, but does cover the use of IT to implement pedagogical decisions and the interface between assessment content and IT delivery. The quality of the assessment content and the scoring procedures are of crucial importance to the validity and reliability of the final result, but fall outside the scope of this Standard. Compliance with this Standard does not indicate that the assessment is pedagogically sound. This Standard should not be interpreted as reducing the need for assessment sponsors to follow good practice in developing assessments, including standardisation and attention to validity and reliability. This Standard does not cover scoring by human markers, but does cover: IT scoring which may be subject to later review by human markers; the IT transmission and delivery of assessments partly or wholly to be scored by human markers and the return of the resulting responses for scoring. This Standard covers the use of IT to provide automatic feedback and instant results, but not result-determination requiring human decision-making or intervention. 1.5 Focus on principles The aim of this Standard is to set out principles and good practice, but not the details of the means by which they are to be achieved. It is therefore possible to follow the recommendations using a variety of technological or procedural approaches. This Standard is not specific to any particular hardware or software platform. In many areas the principles outlined in this Standard will be supplemented by the specific regulations of assessment distributors. 1.6 Compliance Assessment sponsors, assessment distributors and assessment centres may claim compliance with this Standard if they comply with all the clauses or subclauses applicable to their role (see table below). Notes to the clauses indicate the role(s) to which each clause or subclause is applicable. This Standard is applicable to both high-stakes and low-stakes assessments, but some clauses or subclauses are applicable only to high-stakes assessments; this is indicated in Table1. Table1 Application of ISO/IEC23988 Role Assessment type Relevant clauses or sub-clauses Assessment sponsors High-stakes and low-stakes 5.1 Assessment distributors High-stakes 5.2, 6 to 12 inclusive Assessment distributors Low-stakes 5.2, 6.1, 6.2, 6.3, 6.5, 7.1, 7.2, 7.3, 7.4, 7.5, 8.1, 8.2, 8.3, 8.4, 9.1, 9.2.1, 10.1, 10.2, 11.1, 11.2, 12.1, 12.2 and 12.3. Assessment centres High-stakes 13, 14, 15, 16 and 17 inclusive Assessment centres Low-stakes 13.1, 13.2, 13.3 and 13.4 The scenarios given in annexA illustrate how different types of organisation might need to comply with different clauses of this Standard.