Assessment Practices in Local Agencies: Initial and Extended Assessment Activities, Lecture notes of Accounting

Download Assessment Practices in Local Agencies: Initial and Extended Assessment Activities and more Lecture notes Accounting in PDF only on Docsity!

Department of Health and Human Services

OFFICE

INSPECTOR GENERAL

JOBS SKILLS ASSESSMENT

:v'

Richard P. Kusserow

INSPECTOR GENERAL

JUNE 1992

OFFICE OF INSPECTOR GENERAL

The mission of the Offce of (^) Inspector General (DIG), as mandated by Public Law 95- 452 as amended, is to protect the integrity of the Department of Health and Human Services (HHS) programs as well as the health and welfare of beneficiaries served by those programs. This statutory mission is carried out through a nationwide network of audits investigations (^) , and inspections conducted by three DIG operating components: the Office of Audit Services , the Offce of Investigations , and the Offce of Evaluation and Inspections. The OIG also informs the Secretary of HHS of program and management problems, and recommends courses to correct them.

OFFICE OF AUDIT SERVICES

The OIG' (^) s Office of Audit Services (OAS) provides all auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit work done by others. Audits examine the performance of HHS programs and/or its grantees and contractors in carring out their respective responsibilities and are intended to provide independent assessments of HHS programs and operations in order to reduce waste, abuse, and mismanagement and to promote economy and effciency throughout the Department.

OFFICE OF INVESTIGATIONS

The OIG' s Office of Investigations (01) conducts criminal , civil, and administrative investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries and of unjust enrichment by providers. The investigative efforts of 01 lead to criminal convictions, administrative sanctions, or civil money penalties. (^) The 01 also oversees State Medicaid fraud control units which investigate and prosecute fraud and patient abuse in the Medicaid program.

OFFICE OF EVALUATION AND INSPECTIONS

The DIG' s Offce of Evaluation and Inspections (OEI) conducts short-term management and program evaluations (called inspections) that focus on issues of concern to the Department , the Congress, and the public. The findings and recommendations contained in these inspection reports generate rapid , accurate, and up-to- date information on the efficiency, vulnerability, and effectiveness of departmental programs. This report was prepared in Dallas under the direction of Regional Inspector General Ralph Tunnell and Deputy Regional Inspector General Chester B. Slaughter. Project staff included:

Dallas Headquarters Carol L. Cockrell, (^) Project Leader Susan Hardwick Elsie Chaisson Susan Rousseau O' Connell

To obtain a copy of this report, call the Dallas Regional Offce at (214) 767- 3310

EXECUTIVE SUMMARY

Purpose

The purposes of this report are to describe the assessment (^) process and identify useful assessment practices in the Job (^) Opportnities and Basic (^) Skills Training Program (JOBS).

Background

The primar (^) goal of the Famly Support Act of 1988 (^) (Public Law 100- 485) is to decrease dependency on the Aid to Famlies with Dependent (^) Children (AFC) (^) grant by providing education (^) , training and employment (^) opportnities for (^) grantees though the JOBS program. States are required to conduct an "initial assessment" of JOBS parcipants (^) ' work readiness and employability. The results of this assessment are (^) used to constrct the "employability plan " which specifies a permanent employment objective with goals and activities leading

. up to that objective.

Federal regulations provide general guidance and few requirements for assessment activities , leaving the specifics of form and content to States (^) ' discretion. The regulations do require the "initial assessment" be based on the person (^) s famly circumstaces, prior work experiences, job skills (^) , and educational and support services needs. Assessment methods may include testing, interviews, counseling, and the use of self-assessment instrments. If (^) a State tests for literacy, basic skills, aptitudes, and so on (^) , the regulations encourage the State to use "nationally recognized (^) , stadardized (^) , or industr-developed" instrments in order to avoid inappropriate placements and misspent funds.

Methodology

States which have " (^) mature " programs were chosen intentionally, " (^) mature (^) " being defined as having a lengthy operating history (e. (^) , having had the regular Work Incentive (^) (W) program or (^) WI Demonstration through most of the 1980s) and a track record of offering a fairly comprehensive range of services. (^) We selected five States and, with the help of State- level JOBS administrators , selected two sites withn each one. (^) We conducted site visits in the States of Californa (San (^) Bernardino and Yuba counties), Michigan (Bay and Kent counties), Massachusetts (Hampden and Worcester counties), and Nebraska (the Southeast Distrct and the Omaha Distrct). (^) In the State of Florida, telephone interviews replaced site visits at the Ft. Meyers and Orlando Distrcts. (^) At each site, strctued interviews were conducted with the local administrator, two experienced line supervisors and two experienced case managers or equivalents.

Conclusions (^) The Current State of JOBS Skills Assessment

Skils assessment is stil in an early phase of development. Under very (^) general Federal guidelines, local welfare agencies are experimenting with diverse approaches to assessing the job skills of paricipants. (^) A varety of assessment tools, events and practices are being used within a process on which uniform agreement as to its exact content and (^) sequences is still lacking. Thus, it (^) would be premature to identify specific tools and practices as the most effective and efficient, or the most valid and reliable.

Some principles and useful practices are emerging. Even at this early date, it is possible to make several observations about the nature of assessment in local agencies and to describe some of its more regularly occurrng stages and events. It is (^) not, for example, an event which transpires at a single point in time. Rather (^) , assessment is an ongoing process continuing throughout the (^) parcipant's tenure in the JOBS program. Its content not limited to skills testing, but (^) encompasses a varety of formal and informal (^) activities and decisions. (^) Assessment is at the hear of three major functions performed by case managers for the local agency: 1) identifying the parcipant's (^) employment strengths and weakesses , 2) tailoring the service strategy to meet the (^) parcipant's needs, and

3. monitoring progress toward the participant's employment goal.

More experience and research are needed. For this (^) report, we have developed a description of the assessment process from data collected at ten " (^) matue" sites. We have discussed assessment as several sometimes discrete (^) , sometimes overlapping, stages or sets of activities: initial assessment (^) , a preliminar labor market sort, extended assessment (^) , the employabilty plan (^) , and job readiness. We believe this descriptive report provides useful guidelines for practitioners in the field and (^) wil also be useful to those formulating a research or evaluation agenda in the area of JOBS skills assessment.

Departental Comments

We are appreciative of comments received from the Administration for Children and Famlies on the working draft of ths report.

INTRODUCTION

Purpose

The purposes of this report are to describe the assessment (^) process and identify useful practices in the Job Opportunities and Basic Skills Training Program (JOBS).

Background

The Federal Context for the (^) JOBS Program

The primar goal of the Famly (^) Support Act is to increase famly self-suffciency and to decrease dependency on the Aid to Families with Dependent Children (^) (AFC) grant by providing AFDC recipients the opportunities to upgrade their work skills or (^) lear new skills as paricipants in the JOBS (^) program.

Title II of the Famly Support Act (Pblic Law 100-485) establishes the JOBS program under title IV-F of the Social Security Act.! JOBS is the operatig welfare-to-work program in the Deparment of Health and (^) Human Services , replacing and expandig

current authority for education, training, and work programs in title IV - A (AFC) and title

IV -C (Work Incentive (WIN) program and WIN Demonstration) of the Social Security Act.

States were required to implement the program by October 1 (^) , 1990. By October 1, 1992, the program must be available on a statewide basis where feasible, accounting for the number of eligible parcipants, the condition of the local economy, and other relevant factors. The JOBS program in every State must include a varety (^) of educational activities, job skills training, work experiences, job readiness activities, job development and job placement activities , as well as support (^) services (e. , transporttion and child care). The State s IV- A agency (welfare agency) has the responsibilty for the admstration (^) , or the supervision of the administration, of the JOBS program and may contract out varous program activities.

In regard to assessment activities (^) , States are required to conduct an " initial assessment" of JOBS parcipants ' work readiness and employability. Federal (^) regulations provide general guidance and few requirements for the " (^) initial assessment " allowing States to determne specifics of form and content. The (^) regulations do require the initial assessment to account for the paricipant's family circumstances , prior work experiences (^) , current job skills , and

Federal Register, October 13 , 1989, Par II, DHHSIFSA, 45 CFR Par 205 et seq..

educational and support services needs. Assessment (^) methods may include testing, interviews , counseling, and the use of (^) self-assessment instruments (^) If a State tests for literacy, basic skills, aptitudes (^) , etc., the regulations encourage the State to use "nationally recognized , standardized, or industr-developed" instrments before placing (^) JOBS paricipants in order to avoid inappropriate (^) placements and misspent funds. (^) No time period for completing the assessment is specified except that it must be done "withn a reasonable time frame prior to paricipation (in the JOBS (^) program). ,,

The regulations also require the creation of an "employabilty plan (^) " based on the data collected from initial assessment (^) activities. The employability plan is the action (^) plan for the parcipant. It (^) must state a permanent employment objective or outcome, describe the activities the paricipant (^) wil complete (e. (^) , complete high school, go though nurse aide training), and specify any support (^) services to be provided. It must also account for the participant's preferences and needs to the extent allowed by available resources and local job opportunities. The regulations encourage (^) , but do not require, States to include a schedule of activities (^) necessar to achieve employment by a specified date.

Scope

For ths report, we focus on overall assessment practices inside the local agency and do not review specific assessment activities and tools in the education, job training, and the individual or group job search (^) component.

Methodology

Learning About Assessment

Given the tremendous flexibility in the Federal regulations (^) , we anticipated that States and localties would be faily (^) diverse in conceptualizing, settng up and managing the assessment of participants (^) ' job skills. In order for (^) us to fill in the conceptual gap (^) between the broad policy statements in the regulations and the actual (^) implementation of assessment in local agencies (^) , we performed four tasks.

First, we reviewed current literature on welfare-to-work programs (^) , federally supported training programs (^) , assessment-related activities in these programs, and literature on skills testing.3 Relative to the JOBS program, we began to understad (^) assessment as:

Federal Register (^) ibid. p. 42178. 3 See APPENDIX A for a list of sources.

already laid the groundwork for the JOBS program would enable us to immediately benefit from their experiences in integrating assessment into the program.

In a previous Office of Inspector General study on JOBS implementation practices, the mature " criterion was used to select a purposive sample of eight States (^) , diverse in their program strctures , demographic factors (^) , economic conditions and geographical location. We selected five States from ths eight-State sample.

In each selected State , we asked the JOBS administrator to select two local welfare agencies (each in a different county) which had a lengthy operating history and excelled in most or all JOBS program activities (^) , especially assessment. We conducted site visits in the States of California (San Bernardino and Yuba counties), Michigan (Bay and Kent counties), Massachusetts (Hampden and Worcester counties), and Nebraska (the Southeast District and the Omaha Distrct). In the State of Florida, telephone contact replaced site visits at the Ft. Meyers and Orlando district offices.

At each site , we conducted strctured interviews (^) with the local administrator, two experienced line supervisors, and two experienced case managers or equivalents. (^) Data collection activities occurred in September and October of (^) 1990.

We also reviewed copies of available State and site-specific policies, technical assistance and procedures guides regarding assessment; formal and informal interagency agreements governing the provision of assessment services; current JOBS program component and paricipant flow chars; any (^) studies or formal evaluations of the assessment system; and participation/outcome data.

7 "Working Toward JOBS, (^) " OEI- 12- 90- 00960 , June 1990.

THE (^) CONTEXT OF (^) JOBS

The nature and extent of assessment (^) activities inside the agency are influenced by a (^) varety of factors internal and external to the agency. The interaction within (^) and between internal and external factors creates a complex, dynamic environment for the JOBS program and the assessment process.

The External Environment

Federal laws, regulations and guidelines set the stage for the JOBS program and assessment activities. Another set of factors (^) the health of State, regional and local economies, and the politics surrounding welfare (^) programs and their funding (^) constitutes the infrastructure of the external (^) environment. Availability, coordination and delivery of education , employment and job training services at the State and local levels are other factors influencing the nature and extent of the JOBS program and assessment (^) activities.

Local and Regional Economic Conditions Form the Infrastructure

As States and local agencies continue implementing the JOBS program and assessment activities (^) , some external factors are more likely to affect decision-makng than other factors. Because of the very nature of the program (^) , local and regional economic conditions greatly influence the degree of its success. The (^) strategies employed to manage these sets of conditions (^) var from one local agency to another. For example (^) , given the economic conditions of seasonal work and limited diversity in community or regional industres (^) , a local agency might choose the strategy of implementing a more " (^) streamined" JOBS program and assessment process (i. (^) , very limited use of formal assessment activities and staff time). In (^) a few Florida locales, the majority of JOBS parcipants (^) are diected into jobs (^) supporting the seasonal tourism industr, e. (^) , food service , maitenance and repair, domestic service, clerical and sales jobs.

Another example of economic conditions influencing decisions about the JOBS program and the extent of assessment activities is the case of Yuba (^) County, Californa. The county has among the highest unemployment (=14%) and (^) povert (=16%) rates in the State. Although seasonal agrcultural work is available, there are also available positions from year-round employers in retail sales, service industres and (^) government. Rather than focus on seasonal jobs (^) , the administrator has chosen the strategy of increasing the competitiveness of JOBS paricipants for the scarce (^) , above-minimum-wage jobs in the area. For this welfare agency, the (^) strategy has had the effect of emphasizing the importce of (^) the assessment process in detecting the skill proficiencies and (^) deficiencies of paricipants, identifying and dealing with barers to employment, and tracking progress as

paricipants prepare for employment. The agency s commitment to this parcular strategy

commitment to JOBS and to the use of every community (^) resource in meeting the goals and challenges of the program. The (^) administrators, supervisors and case managers who were also supervisors or caseworkers during the now defunct (^) WI program (most of our interviewees) viewed the JOBS program as a substative (^) change for the better. All of them had witnessed positive changes in paricipants (^) ' lives as a result of the program which (^) , in turn , strengthened their commitment. However, they were concerned that the JOBS program and their work with recipients were being thwared by (^) shrnkng budgets for support services (e. (^) , child care and transporttion) and other (^) contracted services (^) , and by heavy caseloads (the average is =100 JOBS clients per case manager), resulting in less time to spend with each paricipant.

Case Management and Specialized JOBS Units Strengthen Program Performance

Staff concerns about tight budgets and heavy caseloads seemed to be mitigated somewhat by two staff arangements working in (^) tandem: a (^) specialized JOBS unit and case management. Not all the (^) agencies we visited had organized a special JOBS unit, and not all had case management. In agencies which had both, the arangement (^) seemed to promote consistency in JOBS policy and procedures (^) , case managers who were efficient information and community services specialists (^) , and consistency and continuity of paricipants (^) ' contacts with the agency.

A Data Processing System Saves Time and Effort for All (^) Agency Staff

Adequate data collection and information synthesis appeared to be vital for identifying, documenting and managing parcipants (^) ' employment (^) barers and strengths. We noted that most local agencies in our sample had installed data (^) processing systems to help case managers and supervisors organize the bulk of the information about (^) parcipants. Agencies with more basic systems could produce " (^) caned" (^) reports (e. , quarerly reports which have a fixed format (^) the same varables and the same data manpulations for every report), while agencies with more sophisticated (^) (and more expensive) systems could produce reports talored to fit the (^) parcular needs of different staf members.

A user- friendly, readily available data processing system seemed to be a valuable tool for saving case managers (^) , supervisors and administrators significant time and effort in compiling and aggregating data about parcipants. (^) It also greatly assisted in monitoring parcipants' progress through JOBS components and activities (^) , cutting down on the "paper shuffle. "

INITIAL (^) ASSESSMENT ACTIVITIES

Typically, within the week after the JOBS orientation session, the parcipant met with the case manager to begin the initial (^) assessment. Using a mixture of formal and informal tools and practices, the case manager collected and examined a body of data about the participant in order to judge the extent of "job readiness. " This included assessing whether the paricipant had adequate basic education skills (especially language and math skills) to satisfy potential employers ' requirements; the extent to which the (^) parcipant's vocational interests and aptitudes (^) , employment history, work experiences and current skills matched labor market demand; whether likely jobs would adequately support the parcipant's family; whether (^) the paricipant had appropriate work attitudes and behavior; and the type and extent of support services (^) needed.

Case Managers Perform Critical Tasks

We found the case manager (^) s principal tasks during the initial assessment were 1) to identify and document (^) barers to employability, as well as the individual' s parcular strengths, 2) to decide which (^) barers can be managed withn the strctures of agency resources and the individual's life circumstances, and 3) to create a workable strategy of agency service delivery and (^) parcipation by the individual. Withn the operational framework of these principal tasks , the case managers performed several ongoing roles as needed , such as:

CASE MANAGER' S ROLES VIS- VIS THE PARTICIPANT DURG THE INITL ASSESSMENT

Establishing a " parership" with the parcipant, and ensuring the individual knows the rules of the parership, i. , the rights and the responsibilities of parcipation in the program; Giving the parcipant as much control as possible over decisions afecting herlhs life; Building the parcipant s self-confidence and urging herlhm to make the most of the opportunities offered by the program; Makng the paricipant aware of the (^) skills and employabilty strengths shele aleady has; Discussing employment possibilties with the parcipant while considerig the parcipant expressed vocational interests and labor market demands; Encouraging the paricipant to set employment goals which are realstic, i.e., appropriate to the parcipant s abilities and famly financial responsibilties; Requesting the (^) parcipant to investigate specific vocational choices before deciding on education and/or work training goals and activities in the parcipant s employabilty plan; Working with the parcipant to determne support services; Creating the employability plan with the (^) parcipant in the spirt of the JOBS parership.

In performng these tasks , the case manager built crucial rapport with parcipants, helping to anchor their commtment to the (^) program and motivating them to complete it.

Tailoring Service Strategies for Special Target Groups Has Intended and Unintended Benefits

The case manager s second and third tasks, determning affordable and workable service strategies , can be diffcult for paricipants with multiple barers.^ In^ some^ instaces,^ these tasks have been made less arduous by recognizing that distinct groups of AFDC grantees

each have their own cluster of barers requiring a fairly predictable aray^ of services.

Young parents are an apt ilustration. Because of their inexperience in the world of work parenting and adult relationships, the strategy for young parents (under the age of 24) typically involves a combination of the following services.

SERVICE STRATEGY FOR YOUNG PARENT

Assistace in obtaining child support from absent parents; Treatment for a drg or alcohol addiction, if needed; Finishing the high school degree; Vocational interests and aptitude testing, and career counseling; Obtaning child care and transporttion assistace; Vocational education and/or job training; Receiving instrction in social skills and instrmenta^ tasks, e.^ g., how to parent, household management and budgeting, increasing self-esteem and self-confidence, realstic plang^ and goal settng, problem solving, managing stress, building and maintaning heathy relationships with adults , work maturity skills and other pre-employment prepartion.

JOBS programs in States as diverse as Nebraska and Massachusetts have made it one of their priorities to address the specific cluster of barers^ faced^ by AFDC recipients who are young parents. They have contracted to develop and implement special programs such as Operation Bridge in the Omaha distrct (NE), the Young Parent Program in the Southeast

distrct (N), and the Pregnant and Parenting Youth Program in Hampden County (MA).

These programs usually furnished individual and group instrction^ and^ counseling^ for several weeks on a varety of problems and issues faced by young parents and workers. Though case notes, formal reports, telephone contact and in-person meetings, contractors provided input to case managers about identified barers, progress^ made by each parcipant, and any recommendations for furter action or specific services.

Beyond the intended, formal purposes of specialized service programs for taget^ groups, some unintended benefits also may occur as a result of these programs.^ For^ example, some Nebraska staff members noted that a positive by-product for parcipants who attended meetings for young parents was their furter empowerment as a result of creating their own information networks, informal support groups, and friendships. These informal groups tended to acquire a life of their own, enduring beyond the confnes of the program.

Informal groups/networks can provide valuable support for parents who are Informal support groups can reinforce JOBS finishing the high school degree, for messges by convincing members to adopt^ certin

those tang vocational education

values and expectations, such as:

courses, and for those in a job training VALVES (^) EXPECTATIONS FOR BEHAVIOR regimen or looking for work. Very often, these groups can help members Education^ get the GED at least develop a feeling of community or (^) Work get training (^) learn as many belonging with persons similarly (^) skils as time permits (^) secure circumstanced and struggling for a (^) the best available job (^) do the better life. They also may strengthen (^) best you can and always strive certain constrctive ideas and behavior (^) for improvement and can help members overcome a sense of alienation from the Self-suffciency^ don^ t depend on government mainstream. (See text box. programs for financial support be personally responsible for supporting your family

Although the Initil Assessment Is Usually an In- (^) House Activity, Some Agencies Utilize Contractors

Although case managers in most agencies performed the initial assessment in-house, some agencies contracted with a business or an outside agency for these services. For example the local agency in Kent County, Michigan, contracted with the Career Development Center to provide initial assessment activities. In the initial assessment, a battery of tests was administered to all paricipants (^) , such as the T ABE (Tests of Adult Basic Education), specially normed for adults, used by the Center to measure reading and math skills; the GATB (General Aptitude Test Battery) for general (^) learng abilty (^) and dexterity; the Hall Occupational Orientation Inventory for value priorities; USES Interest Inventory for work interests; and explorations of the individual' (^) s current skills, employment (^) barers, personal strengths , and career goals. (^) Results were reported to the agency and to a ITP A (Job

Traing Parership Act) vocational counselor who then worked with the JOBS parcipant

to develop an employabilty plan.

Most Agencies Limit or Omit Formal Testing During the (^) Initil Assessment

In contrast to Kent County, Michigan, other local agencies either elected not to conduct formal testing in the initial assessment (^) , or they did so to a lesser extent by choice or by requirement. The State of California, for example (^) , required counties to administer a reading test and a math test from the criterion-referenced CASAS (^) (Comprehensive Adult Student Assessment System). In Yuba and San (^) Bernardino counties, CASAS tests were administered and scored during the JOBS orientation session. Test results added

The Conclusion of Initil^ Assessment Activities

At the end of the initial assessment stage, either a temporar or^ permanent employability plan was created to guide the diverse paths of JOBS paricipants. Some paricipants went on to an extended assessment. Others began education component activities by^ pursuing education remediation , an ESL course, the GED certficate, an Associate^ s^ degree^ or series of vocational education courses. Some paricipants entered the job training component. Certain paricipants were directed into a specialized service program address specific employment barers. Parcipants^ judged as "job ready^ " were sent to the job search component (a " labor market sort

Other agencies used an assessment-like^ practice, which we call the preliminary^ labor market sort , wherein all or most paricipants were^ required to go though an individual group job search immediately upon entering the JOBS program, sometimes without an initial meeting with a case manager. Only parcipants remaining jobless after completing these activities were allowed to take par in^ other JOBS components. The^ advantages and disadvantages of this paricular labor market sort are discussed next.

THE PRELIMINARY LABOR MARKET SORT

We found that the preliminar labor market sort (our term) to be a mandatory, front-end job search which sifts those who are "job^ ready^ "^ from^ those^ who^ are^ not.^ In^ agencies employing ths practice, all or most JOBS parcipants are given job-search training and are required to search for employment as either their first formal activity after an initial meeting with a case manager , or their second formal activity after completing a remedial course in adult basic education , an ESL course or the GED.^ Varously^ named^ Job^ Club Job Workshop, Group (or Individual) Job Search, and so on, it functions as an assessment practice by testing parcipants' current abilities and skills against labor market demand very soon after their entrance into the JOBS program.

The Preliminary Sort Is an Assessment Practice Having Advantages and Disadvantages for the Agency and Partcipants

Because of its mandatory nature and early placement in JOBS, some controversy surounds ths fitering technique. Similar to its operation in the former welfare-to-work program (W), the preliminar sort^ works to the advantage of the local agency and "job ready parcipants having work skills competitive enough to ear wages above the minimum. However, it seems to further disadvantage those who are not "job ready " and those with less competitive work skills the most needy of all JOBS paricipants. The advantages and disadvantages are as follows.

ADVANTAGES OF THE PRELIMINARY LABOR MARKT SORT

To the benefit of the local agency, it has the potential of puttng a signifcant proporton of parcipants to work immedately. To the extent that ths occur, it can initialy (1) decreae the program costs of investing in education and training for those JOBS parcipants who fmd jobs, (2) decrease or eliminate AFC payments for those same parcipants, and (3) increae the "success

rate " of the JOBS Progr.

For parcipants who get jobs, it has the potential of increasing self-esteem , reinforcing the positive aspects of being self-sufficient, thus mitigating the "welfare stigma.

It educates all or most JOBS parcipants ealy in the^ progr^ in tehniques for fmding a job^ and getting hired, and in the atttudes and behaviors valued by employers.