The Ohio State University • College of Education and Human Ecology
Center on Education and Training for Employment
Centergram IssueSummer 2006To subscribe/unsubscribeplease follow this link and fill out the form. PDF VersionClick here to read this newsletter in PDF format. Past IssuesClick here for past issues.
CETE Research Grants Awarded
Three research proposals were selected to be funded in the newly established CETE Research Grant program. The proposals were selected as part of a competitive process facilitated by the Office of the Associate Dean for Research, College of Education. The goal of the program is to provide internal support for scholarly efforts that have future external funding opportunities. Thus, the criteria for selection included both the rigor of the proposal and the potential of the research for future funding. Ronald Jacobs, director of CETE, stated: “The CETE Research Grant program is meant as an investment to continue and enrich the intellectual vitality and relevance of CETE. Our perspective is that there is no substitute for doing quality research when the workforce development field requires practical information.” The funded proposal titles and researchers are the following:
Reports of these research projects will be presented and discussed in an open forum during November 2006. The reports will also be posted on the CETE website. The submission date for the 2007 competition will be announced soon. For more information about the CETE Research Grants, contact Ronald L. Jacobs, director, CETE, at jacobs.3@osu.edu or 614/292-9943. The Use of Job Analysis Data in Constructing Tests of Job Knowledge Tests of job knowledge are used in a wide variety of settings, such as employment (for personnel selection, promotion, placement, and evaluating learning effectiveness or deficiencies); education (secondary and postsecondary career-technical education); and credentialing (licensure and certification). One requirement common to all of these settings is that the tests should be job or occupation related. To establish job-relatedness, the test specifications or test blueprint need to be linked to data derived from a job or practice analysis. This requirement is supported in professional standards that provide guidance on the development of occupationally related assessments, such as the Standards for Educational & Psychological Testing (1999), Principles for Validation & Use of Personnel Selection Procedures (2003), Uniform Guidelines for Employee Selection Procedures (1978), and the NCCA Standards for the Accreditation of Certification Programs (2005). Job analysis data are the most common basis for establishing content validity of job-related tests. A job analysis or practice analysis is the foundation of the test. It defines the content domain or body of knowledge, yields evidence for interpreting scores, and provides evidence for validity. An important consideration is that the method of job analysis chosen should fit the purpose of the job knowledge test. Sample methods include DACUM (often used to develop training and curriculum), WRIPAC (often used to develop selection tests), Job Profiling (ACT Job to Work Keys maps), SHL Work Profiling System (standardized, computerized approach), O*NET (general occupational focus), and the Position Description Questionnaire (PAQ, standardized approach). Practice analysis (Raymond, 2001, 2005) is often distinguished from job analysis in that it is an examination of practice across the occupation rather than a specific job in a specific organization. A task inventory approach such as DACUM or WRIPAC is commonly used in test development. A decision point in the use of job analysis data is whether or not to use the tasks, or instead use the knowledge, skills, and abilities (KSAs) in linking job analysis results to test content. Wang, Schnipke, and Witt (2005) advocate the use of KSAs in establishing the linkage. CETE staff often use a combination of tasks and the KSAs linked to those tasks in developing tests. The goal is to develop a test specification that indicates the “weight” assigned to the content to be tested. For various reasons, the surveys used to gather the data generally focus on the tasks rather than the KSAs (Raymond, 2005). The data are gathered through task verification surveys in which job incumbents are asked to indicate the relative criticality of each task to the job. A number of questions may be asked on the survey to measure task criticality. Most often overall importance of the task to the job is one rating. Another common rating is frequency at which the task is performed. Other possible ratings include needed at job entry (yes/no), level of responsibility, task difficulty, and consequences of deficient performance. CETE staff typically use task importance and task frequency, each on a 0-5 (6-point) scale. A “criticality” variable is calculated by multiplying each individual’s response to these rating items creating a within-person criticality indicator for each task. The individual’s criticality variable is then averaged across individuals to provide the task’s criticality “weight.” Table 1 illustrates how these derived criticality variables are used in creating a proportional number of items for one task cluster or duty: Determine Customer Needs. The criticality is summed across all tasks. Each individual task criticality is divided by this sum and multiplied by 100 to create the percentage of criticality carried by the task. This percentage is then multiplied by the total number of items desired, to derive the number of items to be written for that task. The number of items for each task of a hypothetical 40-item subtest is given in the right column of the table, ranging from 2 to 10. The tasks, along with their linked KSAs and the number of items required, are used by the item writers and facilitators in the test specifications. Multidimensional test specifications can be created by the addition of other factors, such as various cognitive levels required by predetermined proportions of test items.
References Brannick, M. T., & Levine, E. L. (2002). Job analysis: Methods, research, and applications for human resource management in the new millennium. Thousand Oaks, CA: Sage. Raymond, M. R. (2001). Job analysis and the specification of content for licensure and certification examinations. Applied Measurement in Education, 14, 369-415. Raymond, M. R. (2005). An NCME instructional module on developing and administering practice analysis questionnaires. Educational Measurement: Issues and Practice, 24(2), 29-42. Sackett, P. R., & Laczo, R. M. (2003). Job and work analysis. In W. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Handbook of psychology (Vol. 12, Industrial/Organizational Psychology, pp. 19-37). New York: Wiley. Wang, N., Schnipke, D., & Witt, E. A. (2005). Use of knowledge, skill, and ability statements in developing licensure and certification examinations. Educational Measurement: Issues and Practice, 24(1), 15-22. The OSU South Centers Business Development Network and the Center on Education and Training for Employment have been awarded a research grant from OSU CARES to revise and update the well-received PACE (Program for Acquiring Competence in Entrepreneurship) curriculum. The new version will be available by January 2007. Watch for further announcements. ISO 10015 Planning Meetings, July 18, 2006 (AWE representatives), July 19 (companies). Planning meetings for organizations interested in the CETE/ISO 10015 Development Program. For information: Ron Jacobs, jacobs.3@osu.edu, 614/ 292-9943. TtT Summer Institute Sponsored by the Ohio Transition to Teaching project. For information: Belinda Gimbert, gimbert.1@osu.edu, or Maria Hruby Moore, moore.1149@osu.edu. Test Construction Workshop Constructing valid, reliable assessments of job-specific knowledge and skills is critical to effective human resource practice. This workshop provides participants with practical knowledge and skills to understand the creation of valid, reliable, and legally defensible assessments. For information: James Austin, austin.38@osu.edu, 614/292-9897; or Robert Mahlman, mahlman.1@osu.edu, 614/292-9072. DACUM (Developing a Curriculum) Institute, July 10-14, 2006, 8am-5pm, $1,395. DACUM Institute, September 11-15, 2006, 8am-5pm, $1,395 DACUM Institute, November 6-10, 2006, 8am-5pm, $1,395 SCID (Systematic Curriculum Instructional Design) Workshop, July 17-21, 2006, 8am-5pm, $1,195 SCID Workshop, September 18-22, 2006, 8am-5pm, $1,195 SCID Workshop, November 13-17, 2006, 8am-5pm, $1,195 All are hosted by CETE/OSU, Columbus, OH. For information: Robert Norton, norton.1@osu.edu Work and Family Life Resource Guides Personal Development Resource Guide—WFL-01, $35 Focuses on practical problems related to taking responsibility for self and others, building self-esteem, building relationships with family and peers, managing stress and conflict, planning for future careers, and making responsible choices regarding sexuality and parenting. Includes 3-ring binder. Resource Management Resource Guide—WFL–02, $35 Focuses on practical problems related to managing resources to achieve personal and family goals; making informed consumer choices; creating and maintaining a living environment; selecting, obtaining, and maintaining clothing; making food choices; and preparing and serving nutritious food. Includes 3-ring binder. Life Planning Resource Guide—WFL-03, $35 Focuses on practical problems related to developing a life-management plan, caring for self and others, building and maintaining constructive interpersonal relationships, building and maintaining strong functional families, developing strategies for lifelong career planning, coordinating personal and career responsibilities, and establishing a plan for using resources to meet individual and family needs and goals. Includes 3-ring binder. Nutrition and Wellness Resource Guide—WFL-04, $35 Focuses on practical problems related to making choices that promote wellness and good health, analyzing interrelationship between food choices and wellness, evaluating relationship between psychological and social needs and food choices, planning strategies for choosing foods that promote wellness, obtaining and storing food, preparing and serving nutritious meals and snacks, selecting and using equipment for food preparation, and identifying strategies to promote optimal nutrition and wellness of society. Includes 3-ring binder. Family Relations Resource Guide—WFL-05, $35 Focuses on practical problems related to the role and significance of family; preparing for adult life and family life; nurturing human development through the life span; building healthy family relationships; managing stress, conflict, and crisis; coordinating work and family; and dealing with the social forces that impact on families. Includes 3-ring binder. Parenting Resource Guide—WFL-06, $35 Focuses on practical problems related to parenting roles and responsibilities; developing readiness and preparing for parenthood; meeting the physical, emotional, social, and intellectual needs of family members; nurturing healthy, caring relationships; parenting in various family structures and cultures; and meeting the responsibilities of families and society in nurturing children. Includes 3-ring binder. To Order: Order
online at http://cete.org/publications.asp
or send your orders to Publications, Center on Education and Training
for Employment, The Ohio State University, 1900 Kenny Rd., Columbus,
OH 43210-1016.
How to Unsubscribe (but we hope you
won't!) Share Centergram with a friend
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
