Improve HR Processes with Test Construction Workshops

Constructing valid, reliable assessments (tests) of job-specific knowledge and skill is critical to effective and defensible human resource practice. Knowledge and skill assessments are used for many personnel-related functions, including—

• personnel selection and placement
• personnel certification and credentialing
• training needs assessment or diagnosis
• training evaluation

A carefully designed, systematic approach to test development maximizes return on investment and contributes to employee or job candidate satisfaction and motivation.

Poorly designed assessments can yield erroneous and damaging results, leading to employee or candidate dissatisfaction, ineffective personnel decisions, and increased exposure to litigation.

Test Development Workshops offered by the Center on Education and Training for Employment at The Ohio State University provide the participant with practical knowledge and skills. Through a combination of discussion, lecture, and exercises, the test development workshop will help the participant—

• understand the creation of valid, reliable, and legally defensible assessments
• evaluate assessments constructed by others

Because job analyses serve as the foundation for any occupationally specific test development process, our test development workshops build on information gathered from job or occupational analysis procedures. Our workshops are designed to complement multiple methods of job and occupational analysis.

We offer a 3-day basic workshop that addresses the following topics:

Test development cycle
Legal issues in employment testing
Information about jobs or bodies of knowledge
Creation/evaluation of test specifications
Reliability and validity
Selecting item writers
Items: development, banking, reviewing
Setting cut scores (pass/fail standards)
Developing test forms
Test administration procedures and guidelines
Pilot testing assessments: Item and test analysis
Interpreting and reporting scores
Maintaining an assessment system
Evaluating own and others’ assessments

The advanced 2-day workshop under development will address additional topics in more detail, including the following:

Advanced test and item monitoring
Advanced demonstration of item analysis
Computer Adaptive Testing (CAT)
Advantages of online and computer-based testing
National and international testing standards
Major test theories
Trends in the testing industry
Overview of major test categories and publishers

Workshop Presenters

James T. Austin, PhD and Erich C. Fein, MA are the workshop presenters. Both have training in psychometrics and experience in large-scale and small-scale test construction.

Jim Austin received his PhD in Industrial-Organizational Psychology from Virginia Tech in 1987 and was a Postdoctoral Trainee in Quantitative Psychology at the University of Illinois. He has been on the staff of CETE since 1997.

Erich Fein, who has worked in the testing field since 1999, received his MA in Industrial-Organizational Psychology in 2001. He will be completing his PhD at The Ohio State University in June 2005.

Three-day basic test construction workshops will be held—

March 7-9, 2005 (see Events for details)
June 6-8, 2005 (check our website’s Events page at a later date: http://cete.org/events.asp)

Please feel free to contact us for registration materials and for further information about the Test Development Workshops.

Robert A. Mahlman
Director of Assessment and Evaluation Services
Center on Education and Training for Employment
The Ohio State University
voice: 614/292-9072; fax: 614/292-4323
e-mail: mahlman.1@osu.edu

Preparing Better Teachers for Tomorrow (PBTT)

PBTT draws from over 30 years of sustained research and development effort by the Center on Education and Training for Employment (CETE). Modules in this series are based on the national identification and verification of the professional development competencies important to career-related teachers and instructors. A 1995 DACUM occupational analysis workshop conducted by Robert E. Norton was sponsored by Illinois, Ohio, and Pennsylvania to identify the duties and tasks believed to be important to career-related teachers/instructors. These competencies were verified by a stratified national sample of 215 career-related teachers/instructors.

The original PBTE (Performance-Based Teacher Education) curriculum packages were products of 18 years of research and development by CETE. Many individuals, institutions, and agencies made contributions to the systematic development, testing, revision, and refinement of those training materials. More than 40 teacher educators provided input into the development of the modules; over 2,000 teachers and 300 resource persons in 20 universities, colleges, and postsecondary institutions used the materials and provided feedback to CETE for revisions and refinement. The first published edition of the modules found widespread use nationwide and in several other countries around the world.

The structure and design of the Preparing Better Teachers for Tomorrow (PBTT) series follows that of the PBTE modules, which addressed the professional competencies needed by career-technical teachers during the 1980s and 1990s. Each module provides learning experiences that integrate theory and application; each culminates with criterion-referenced assessment of the learner’s performance of the specified competency. The materials are designed for use by persons learning to become teachers/instructors who work individually or in groups under the direction and with the assistance of teacher educators or others acting as resource persons. Resource persons should be skilled in the teaching competencies being developed and should be thoroughly oriented to PBTT concepts and procedures before using these materials.

The design of the materials provides considerable flexibility for planning and conducting performance-based training programs for preservice and inservice teachers/instructors as well as business-industry-labor trainers, to meet a wide variety of individual needs and interests. The materials are intended for use by universities and colleges, state departments of education, postsecondary institutions, local education agencies, business/industry, and others responsible for the professional development of instructors and other occupational trainers.

Preparing Better Teachers for Tomorrow (PBTT) includes modules on new topics as well as revisions and updates to the original PBTE module series. The following PBTT modules are available for purchase:

Instructional Execution

Direct Student Lab Experience (107C, $10)
Introduce a Lesson (110C, $7)
Summarize a Lesson (111C, $7)
Employ Oral Questioning Techniques (112C, $7)
Employ Reinforcement Techniques (113C, $7)
Provide Instruction for Slow and More Capable Learners (114C, revised edition coming soon)
Present an Illustrated Talk (115C, revised edition coming soon)
Demonstrate a Manipulative Skill (116C, $7.50)
Demonstrate a Concept or Principle (117C, $8.75)
Employ Computer-Based Instruction (104PT, $9.95)

Academic & Vocational Integration

Prepare to Integrate Academic & Vocational Curriculum (101PT, $8.95)
Implement an Integrated Academic & Vocational Curriculum (102PT, $11.95)
Improve Your Integrated Academic & Vocational Curriculum (103PT, $8.95)

Program Planning, Development, and Evaluation

Develop a Course of Study (108A, $9)

Instructional Evaluation

Establish Student Performance Criteria (101D, $7.50)
Assess Student Performance: Knowledge (102D, $8)
Assess Student Performance: Skills (104D, $7)
Determine Student Grades (105D, $8.75)
Evaluate Instructional Effectiveness (106D, $7.50)

Instructional Management

Provide for Student Safety (105E, revised edition coming soon)
Manage the Vocational Laboratory (109E, revised edition coming soon)

Instructional Planning

Develop Performance Objectives (102B, $7.50)
Develop a Unit of Instruction (103B, $7.75)
Develop a Lesson Plan (104B, $7)

To Order: Prices do not include shipping and handling. Shipping costs are figured at 10% of total order. Priority shipments charged at actual costs plus $10 handling. Call for quantity discount information (614/292-4277). Send orders to Publications, Center on Education and Training for Employment, 1900 Kenny Road, Columbus, OH 43210-1090.

Events

DACUM (Developing a Curriculum) Institute
February 14-18, 2005, 8am-5pm
Hosted by St. Louis Community College, St. Louis, MO
Registration Fee: $1,195

For information: Robert Norton, norton.1@osu.edu
Debbie Weaver, weaver.22@osu.edu
http://www.dacumohiostate.com

Assessment Test Construction Workshop
March 7-9, 2005, 9am-4:30pm
Hosted by Assessment Services CETE/OSU, Columbus, OH
Registration Fee: $750

Covers the following topics: the test development cycle; item writing, banking, and reviewing; basics of item and test analysis; developing test forms; test refinement procedures; content validation; interpreting and reporting scores; setting cut scores (pass/fail standards); maintaining an assessment system; and evaluating tests and testing systems. (See the lead article above for more!)

For information: Jim Austin, austin.38@osu.edu
Erich Fein, fein.5@osu.edu

DACUM Institute
March 21-25, 2005, 8am-5pm
Hosted by CETE/OSU, Columbus, OH
Registration Fee: $1,195

For information: Robert Norton, norton.1@osu.edu
Debbie Weaver, weaver.22@osu.edu
http://www.dacumohiostate.com

All-Ohio Youth Summit to Feature Entrepreneurship

Nearly 7 of 10 teens say they want to run their own businesses in the future, according to the Kauffman Foundation. Research shows that youth who participate in entrepreneurial projects have—

• Higher achievement motivation
• More innovative and creativity
• Higher personal control
• Higher self-esteem
• Better academic performance

Make Your Own Road: Youth Entrepreneurship is the theme of this year’s All-Ohio Youth Leadership Summit, hosted by LearningWork Connection, CETE, on March 9, 2005. The Summit is an opportunity for WIA-eligible youth ages 14-21 (and the adults who work with them) from across Ohio to come together for one day to learn from one another about what it takes to run your own business. Youth will work in small groups with other young people from around the state doing hands-on projects that will stimulate their creativity and empower them to believe they can make their own road to entrepreneurship. Activities for adult chaperones are also planned. Both groups will be inspired by the dynamic keynote speaker, Jason Dorsey, winner of the 2004 Austin Under 40 Entrepreneur of the Year award.

The All-Ohio Youth Leadership Summit will be held at the Fawcett Center, Ohio State University, Columbus. It is sponsored by the Ohio Department of Job and Family Services and supported with federal funds through the Workforce Investment Act of 1998.

There is no fee to attend. Registration materials may be downloaded at http://www.learningworkconnection.org. For information: Michele Stenger, e-mail: stenger.20@osu.edu; 800/848-4815, ext 2-8665.

CETE Publications

History and Evolution of Vocational and Career-Technical Education: A Compilation by Michael E. Wonacott (SN72, $5.50)

This compilation sketches the evolution of the field from vocational education to career-technical education throughout the 20th century in the United States. It includes information and excerpts from a variety of sources, including both original and foundational documents from the early 20th century and more recent histories and syntheses from the end of the century.

Future of Work: Some Prospects and Perspectives. A Compilation by DaeYeon Cho and Susan Imel (SN73, $6)

The question of what the future of work in the United States will be is examined in this publication using current information on trends and issues related to work, the economy, and the labor force. It suggests some possibilities for what might occur in future workplaces. An annotated list of 13 print and 5 web-based resources on the future of work concludes the publication.

Multiple Literacies: A Compilation for Adult Educators by Glynda A. Hull, Larry Mikulecky, Ralf St. Clair, and Sandra Kerka (SN74, $7)

This compilation looks at a broader definition of literacy as the application of critical abilities to several domains of importance to adult life. Glynda Hull examines critical literacy for “challenging times.” Larry Mikulecky considers how people acquire digital/electronic literacy and how adult educators can support this effort. Ralf St. Clair uses environmental literacy as an example of education for effective critique rather than instrumental purposes. Sandra Kerka addresses the importance of health literacy for all adults, not just those with low levels of basic skills. An annotated resource list points to information sources about critical aspects of literacies.

To Order: Prices do not include shipping and handling. Shipping costs are figured at 10% of total order. Priority shipments charged at actual costs plus $10 handling. Call for quantity discount information (614/292-4277). Send orders to Publications, Center on Education and Training for Employment, 1900 Kenny Road, Columbus, OH 43210-1090.

CETE Contacts

How to Unsubscribe (but we hope you won't!)
Send a message to Judy Cohen at cohen.5@osu.edu to be removed from this mailing list.

Share Centergram with a friend
Know someone who might enjoy receiving our newsletters? Send them to our website for information (http://cete.org/CentergramIntro.asp) and let them decide if they would like to subscribe.