The ILS Coalition conference was last week, and it kind of got me going, so there may be a few posts this week. I promised earlier to talk in more depth about evaluation for system and the topic came up at the conference. Folks have waited avidly and long enough for the answers. Hating to disappoint anyone, here they are.
I guess the first question is why we need an evaluation. Here are three reasons:
1. To improve the quality of the system,
2. To eliminate waste in the system,
3. To learn how policy decisions are connected to outcomes.
California's statutes which set out the goals and methods of the system set some criteria for how quality is defined. The system succeeds when it:
A. Provides support for the choices clients make as to how they live and which goals they pursue,
B. Allows people with developmental disabilities to live lives similar to those experienced by their non-disabled peers (this is typically, but not necessarily described as integration of clients into the broader community),
C. Preserves the health and maximize the safety of clients.
Since evaluation becomes the set of incentives that professional participants face, the system that works for California should respond to the needs numbered 1-3, while incenting the mission of the system described in A-C. To accomplish all of this, the following should be elements of a statewide system for evaluation:
Client-centeredness: While the evaluation has to begin with valid outcomes which will make sense in the aggregate, the value of the measurement should depend on the stated goals of the individual client. Some of the waste in our system comes from supporting individuals to accomplish goals that don't interest the client. Sure, I agree, everyone should work and participate in the greater community but its awfully easy to lose an unwanted job or get kicked out of a club you hate to see. The statewide evaluations can and should aggregate the outcomes the clients choose to pursue rather than measure the success of clients in achieving statewide policy goals. I bet I write a better explanation of this another day.
Reasonable Expectations: A baseline level of accomplishment should be predicted and the outcomes judged against that baseline. This doesn't suggest low expectations, it refers to the idea that clients who face greater challenges should be as desirable to serve as people who are more typical of the population at large. No-one can defend planning for people based on traits like measured IQ, assessed functional abilities or behavioral history. Nonetheless to extend proper credit and reward audacity, baseline expectations can and should be adjusted with standard measures of ability. This can not only help make more challenged clients more attractive to serve, but may ultimately allow clients and their family members to identify agencies that may succeed especially well (or fail spectacularly often) with similar challenges to theirs.
Universality: Evaluations should focus on every client every year and reflect on every program they used and the regional center they're with. In order to identify successful strategies, every program should be measured for success and diverse cohorts should be identified. With 200,000 participants an annual survey of client outcomes can quickly separate what is working with whom from what is waste. Furthermore, the potenial exists to make service and agency selection an informed choice which it isn't today. The faster successful programs and strategies are identified and connected to real people, the less waste and failed agencies will tie up scarce resources.
Transparency and availability: The means of assessing success and the scores achieved by agency, program type and regional center need to be accessible to everyone. This is where there's a pragmatic difference between California's private sector system and other states' self-run systems. In a single system, the goal of evaluation is to measure the success of the overall system. California's system depends on the private sector to provide and clients to select the services and supports which will succeed for each individual. The best system for eliminating waste would be to insure that people considering services have the means to identify the right supports and providers through clear, transparent and easily available information.
If you've read this far, I probably owe you lunch.
3 comments:
“California's system depends on the private sector to provide and clients to select the services and supports which will succeed for each individual. The best system for eliminating waste would be to insure that people considering services have the means to identify the right supports and providers through clear, transparent and easily available information.”
Doug,
This is so well put it makes me tremble a bit when I ponder why much of what you say is not a reality. There is a pragmatic difference between a centralized system and a private system. A consumer can of course obtain services outside the catchment area of the providing regioanal centers. However, since this is rare I ask this question.
Can an individual regional center develop and provide, within its own catchment area, what you described?
Can an RC, within its own catchment area, provide for an evaluation to
1. [I]mprove the quality of the system,
2. To eliminate waste in the system,
3. To learn how policy decisions are connected to outcomes?
Can an RC succeed by:
A. Provides support for the choices clients make as to how they live and which goals they pursue [within its own catchment area],
B. Allows people with developmental disabilities to live lives similar to those experienced by their non-disabled peers [within its own catchment area] (this is typically, but not necessarily described as integration of clients into the broader community),
C. Preserves the health and maximize the safety of clients [within its own catchment area].
If it cannot then why not? Does it not have enough resources, monetary or othewise? Does it not have the technical ability and/or know how? If it has enough resources and the ability then what is it that prevents an individual Regional Center from developing an evaluation system that will improve upon the economics and qaulity of services within its catchment area?
As a side note, I have one exception to your post.
“private sector to provide and clients to select the services and supports”
Indead, the private sector (vendors) provide for 98% of the services that are provided. However, it is the Regional Center that selects the servcies and supports that a consumer receives. People with developmental disabilities have the title of “consumer” but the Regional Center by all practical definitions is the consumer of services because it is granted by Lanterman the satutory authority to make placement and service decisions. There are of course many constraints placed upon the Regional Center. It does not have plenary authority. But – the lack of plenary authority does not mean it doesn’t have the greatest authority.
And IPP meeting consists of many components. The Regional Center represenetave, the IDT team, perhaps a provider, and of course the consumer and representatives. The person the most weight at this meeting is indeed the consumer, but the Regional Center is still the chair person. We know that a consumer cannot demand that a supported living arrangement be developed on Rodeo drive when one on Sunset Blvd. Exists and meets the need of the consumer. This is true even if a Rodeo drive provider would “better” suit the needs of the consumer.
Almost 30 years ago a judge opined that, “since the responsibility is [the Regional Center’s] to make the placement decision, clearly the Legislature intended its decision to be effective unless it could be clearly demonstrated to be erroneous. Accordingly, as we view it, the burden of proof rests on the parties opposing [the Regional Center's] recommendation to establish that the recommendation should not be honored.”
This does not include changes in services delivery. Any changes in the status quo places the burden of proof on the Regional Center. However, even in these cases the Regional Center has broad authority in regards to ‘cost effectiveness’ and other categories. ‘Burden of Proof’ may seem like an esoteric legal term that should be avoided. Perhaps, but the understanding of ‘burden of proof’ clearly show who, or what (in this case the Regional Center) has the authority under Lanterman, public sentiment, and paradigms notwithstanding.
This reality is a little known reality, and a club that the Regional centers will never advertise and rarely wield. In fact, the top brass of the Regional Centers are simply unaware of their authority (at least this is what is presented publicly). Regardless, with such statutory authority placed in the hands of the Regional Center it would seem that there is little (aside from the lack of knowledge of the authority) to stand in the way should a Regional Center decide to implement an evaluation protocol and make placement decisions based on that protocol.
paul
ps
you owe me lunch
Paul, thanks for a great read. Better than the post deserves. Since I don't know that we've met in person, should I leave your lunch on a park bench? If so, which one.
To respond to the first set of questions, I would consider it vital that the regional centers not be responsible for measuring outcomes. Maybe it's part of a more useful Life Quality Assessment. The reason for that has to do with the importance of also disciplining POS policies. It would be helpful if we could correlate POS policies with outcomes. I doubt much policy is made in the whole world as ignorantly as regional centers are forced to make theirs.
Regarding your question as to RC success, I think those are the definitions of the mission from the Lanterman Act and I'm very comfortable with those. I think regional centers, like service providers could do better generally and in some cases should do enormously better. I think we know too little about this system to know whether it's underfunded for success. We know it's underfunded for success plus waste. If we could find the waste, we would be more successful.
You are always welcome to take exception with any thing I say, but I don't think we disagree very much where you did today. By tradition and law, regional centers are private entities, even if they sometimes look like fiefdoms and duchies. Your point is exactly why I think useful information about services is useful.
Let's say a client wants help with employment and there are ten supported employment agencies and twenty ILS or SLS vendors who do work around work. Maybe 3 of those 30 agencies would help that client brilliantly, 3 would likely drown the client, and the other 24 are various degrees of so-so. We have no information as to which are which. The SC doesn't have that information. The client has nowhere to go for that information. So the SC uses judgement based on a distant sense. This is why Regional Centers are essentially monopsonies. Now imagine all 60 agencies are evaluated by common criteria as employment assistance and the client, service coordinator and agencies all have access to that information. Now its a little bit more like a marketplace.
Doug said...
“Paul, thanks for a great read. Better than the post deserves. Since I don't know that we've met in person, should I leave your lunch on a park bench? If so, which one.”
Doug,
I understand your position regarding measured outcomes and disciplining the POS policies. Indeed, it may be better for this to occur at the state level. I am unqualified to rebut, but later I will try to toss a hedge or a ditch into the steeplechase for the sake of it…
My understanding and wisdom aside, COULD a regional center engage in measure outcomes and disciplined POS policies within their own catchment area? Consider it a Socratic query. We do not currently have information about your 3-3-24/brilliant-drown-so-so ratio, but could we? Is it possible at the RC level, and if not then why not?
It is interesting to realize that the Regional Centers are both a monopsony and a monopoly. It is the only buyer of services within a catchment area, and it is the only broker of services within a catchment area. Sweeeeaatt job eh! How can you go wrong? Sorry…rather, how can you go wrong and suffer for it?
Another Socratic conceptual scenario/question might go like this. How would the picture change if say we had 100 regional centers with overlapping catchment areas! Each consumer would have a choice of three regional centers. For the sake of argument let us disregard the disruption caused by transitioning to such a playing field, and the resistance from the intransigent beneficiaries of the status quo. For the sake of the argument let us presume the we have developed the perfect system to prevent ad nauseam and unreasonable ‘leap-frogging’ of consumers, and the inevitable monopoly that will develop from natural selection. Just think, crappy Regional Centers will just fad away. They will not be reorganized, or given a vacation by DDS only to return as the same old beast.
In regards to centralized measuring outcomes and Life Quality Assessments I have this question. The original idea behind Lanterman was to create 21 regional centers for 21 million people that were hopefully capable of dealing with local issues and local needs. Can it be argued that measured outcomes and life quality assessments are local needs and therefore should be handled locally? Certainly, the city slicker from Santa Clarita has different needs than the bumpkin from Gerber. Is it not logical to conclude that life quality assessments and the measure of outcomes would be different?
As for lunch – It is obvious that my reputation has not preceded me. Some high quality detritus left near any culvert will square the deal…
Post a Comment