Visible Learning

What works in education? Ah, but how well does it work? No, but how well compared to other things does it work? What are the most effective influences on achievement? What about all the things we wring our hands over that actually make trivial although positive differences (class size) or have no effect (learning styles)?

A work friend of mine, Teletha, put me on to "visible learning," John Hattie's meta-analysis of meta-analyses of effects on education for children and youth ages 4-20. I don't know whether Hattie has employed good criteria for the GIGO problem in meta-analysis, let alone meta-meta-analysis: garbage in, garbage out. This means that you don't want to include bad studies in your meta-analysis, so you have to have inclusion and exclusion criteria for the studies.

His point about a positive effect size is one that many researchers understand, but some policy makers might not (and some researchers don't understand, and some policy makers do): The purpose of an effect size is that it gives you a range of the magnitude of the effect and does not rely on a cut-off point, like the p value associated with the null hypothesis statistical test does. So I don't get too technical, perhaps you should look at this web page for an explanation of Hattie's logic: http://www.learningandteaching.info/teaching/what_works.htm.

He needs to be careful not to use .4 as a cut-off. Nevertheless, the point is well taken, especially when you understand the concept of standard deviations. Hattie has found that the mid-point of effects on educational achievement is .40 (40% of a pooled standard deviation), so effects less than that are relatively weak to the effects above that. And effects are on a continuum with a common metric, so you can tell how much more effective one variable is than another.

Now it's time to show you a couple of 15-minute videos of "visible learning." Part 1: http://www.youtube.com/watch?v=sng4p3Vsu7Y&feature=related. Part 2: http://www.youtube.com/watch?v=lS_AackYwEo&feature=related.

The RBI and Speech-Only Children

Why should we do a Routines-Based Interview for children whose only need is related to a speech or language delay? After all, the RBI is lengthy and addresses much more than the presenting problem. More efficient methods exist for identifying speech or language goals.

Four reasons explain why the RBI is a good practice to use with “speech-only” children. (This colloquial label violates the person-first rule and is used to acknowledge my familiarity with early-intervention team vernacular!)

1. Early intervention is supposed to be more than a rehabilitation or remediation service. The purposes of early intervention are to support the family and other natural caregivers in promoting the child’s development just to provide a service to tackle those things that have gone wrong. It is supposed to address all areas of child development and family functioning, including and especially parenting.

Speech-only children are developing in other areas beyond those identified as deficient or delayed. They are learning to play, to solve problems, to move, to handle increasingly small and complicated objects, to get along with others, and so on. The ticket into early intervention is admittedly an established condition or a delay, but, once in, the program has an obligation not to put on blinders and address only the deficits.

Whether the early intervention system should pay for services addressing typically developing areas of development is a matter of public policy. From child development and family systems perspectives, addressing nondeficit areas is appropriate even for single-deficit children. In child development, to promote children’s language, we know it is helpful to promote their overall engagement. In family systems, we know that family members have many opportunities to work on children’s speech and language throughout every day. Therefore, it is not inappropriate for public funds to be used for promoting skills other than the deficient ones.

Another developmental reason for taking a broader view to needs assessment (i.e., conducting an RBI) is that children with speech or language delays might have behavior problems. Research has documented these areas are associated with each other more often than would be expected by chance. Therefore, we should conduct needs assessments for speech-only children that assess engagement (i.e., appropriate behavior) across the day.

The last issue related to the purposes of early intervention is that speech-only children have parents and other family members who might need support. An effective way of finding out what kind of support they might need is to conduct an RBI. This procedure helps identify whether they need emotional support, material support, or informational support, whereas traditional speech or language assessments do not identify support needs adequately.

2. What’s the best way of addressing a child’s speech-language goals? The most effective and efficient method for providing interventions to young children is to embed interventions into everyday interactions, activities, and routines. To do this, the team of people designing the interventions (i.e., the family and professionals) need to know what happens currently in routines and what the desired behaviors are (even if they are primarily about speech or language). Understanding the concept of goodness of fit, when children have speech or language deficits that means that the demands for communication and social interaction in different routines and the abilities or interests of the child do not match well. In other words, meaningful participation in a routine might require the child to speak clearly and, if the child doesn’t yet have the ability to speak clearly, a functional problem ensues. Now we have three options: (a) change (i.e., teach) the child, (b) change the routine, or (c) change your expectations. This goodness-of-fit approach requires us to assess routines in a way best accomplished by the RBI.

3. Do we really want to discriminate against speech-only children and their families? The RBI is heartily endorsed by families, has been shown to be effective, and is valued by many early intervention programs around the world. To deny families with children with speech or language delays the opportunity to participate in an RBI is discriminating against them, therefore, on the basis of the child’s specific disability.

4. The RBI helps determine the functional needs that arise because of the child’s speech-language deficits. Often the existence of a delay is determined, appropriately, by a norm-referenced measure, so that a person can say with some confidence that the child’s reported or observed speech or language is behind what would be expected for a child of that age. What it does not tell us is how this impairment affects functioning. If it did not affect functioning—if the child lived on a desert island, being raised by friendly animals who did not have conventionalized communication (so we might have to rule out chimpanzees), the deficits might not be important. Who cares? So-called normal communication isn’t needed. But usually there is some functional impact of a speech or language impairment. In fact, often the functional impact is what precipitates a referral to early intervention. To obtain an environmental scan of the current and potential impact of the impairment, an RBI is ideal. It addresses the everyday contexts of the child’s life, which are more varied and challenging than life on a desert island.

Without everyday context, the documented deficits could become “speech” goals that have no relevance to specific times, people, places, or activities. Therefore, children can be working on their final-th sound (e.g., both, bath, Beth, cloth) completely devoid of a reason to be using this sound. Although decontextualized instruction can be used with older children, it is usually very difficult to use effectively with young children. But many therapists and teachers try.

The final issue about functionality is that speech goals, as distinct from language goals, are actually about speaking so as to be understood. People familiar with my model will recognize these as the controversial “artic” goals—where we wonder why so much attention goes into articulation therapy when articulation is not resolved until children are in elementary school. But of course I do recognize that families want to be able to understand children and that children want to be understood when they speak. All children go through artic training by their natural caregivers, but some children still have great difficulty making themselves understood. So understandability goals are very relevant. The RBI can help figure out who needs to understand the child, when, where, and about what. Further assessment, usually by a speech-language pathologist, can help identify what specific problems the child might have, such as a structural or tone problem, which might guide what the intervention options are.

So, when you wonder why you’re going to all the trouble of doing an RBI with the family of a speech-only child, remember it’s because early intervention is supposed to address the whole child, it’s the best way of identifying speech-language goals, it’s discriminatory to deny an RBI to any specific group of families, and it helps determine functional needs.

Parenting Website

Check out raisingchildren.net.au, a terrific website for parents and the professionals who work with them.

Training Practitioners in Interactive Skills

http://sender11.typepad.com/.a/6a00d8341e626f53ef0115712117e1970c-pi

Today, I was corresponding with a colleague about how to train people to take a consultative approach to home visiting and our need to spell out how a person should interact.

Our need to categorize human behavior comes from wanting to understand it and wanting to break it down so we can train others. The problem is that the categorization is retroactive. We look at what people do right and wrong and then categorize the chunks of behavior. The problem is that in interactions the categories run together, categories switch in response to the other person, and you can’t plan for a particular way of using the categories. So the retrospectively derived categories don’t necessarily help with training others, which is the prospective use of the categories. This gets to the two nonmutually exclusive types of checklists—read-do and do-read. The former is like recipes; the latter is like looking over your list just before you check out at the grocery store or you close your suitcase. I’m afraid that checklists for interactions are better as do-read tools, because they came from a retrospective categorization of behavior. This then begs the question of what we should use to prompt use of desired interactional behaviors and avoidance of undesired interactional behaviors. I am currently focusing on very simple rules that the learner can remember going into the interactions—rules that should prompt the desired behaviors, especially when paired with some observation-based feedback on details of the interactions, such as those on a checklist. Examples are incidental teaching of children (engage-follow-elicit-reinforce) and family consultation (pass the ball four times before you shoot/ask four questions before you make a suggestion). By themselves, these are simplistic (which doesn’t mean the same as simple, of course, which they also are). But they are paired with advance knowledge (workshops, readings) and training (observation-based performance feedback). The point is the simple “rules”—not the more complicated tools—are the prompts. These two that I’ve mentioned are supposed to be memorized but that can be assisted with visual reminders of some kind.

In Love With Montana

“I’m in love with Montana. For other states I have admiration, respect, recognition, even some affection. But with Montana it is love. And it’s difficult to analyze love when you’re in it.” 
― John Steinbeck, Travels with Charley: In Search of America

Montana's early intervention system is embarking on a 3-year journey to implement what some people call the McWilliam model, also known as the routines-based early intervention approach. If I loved the state before, which I was beginning to do, I certainly do now!

For years, this was a state I'd never been to, because I pretty much go only to states that invite me. And Montanans saw no need to do that. My friend Mark Wolery, from Montana, had told me about the place--more from the hardscrabble side of things than the romantic side, but he'd also put me on to the writer Ivan Doig, who wrote books about Scottish settlers there. Finally, my wife and I ostensibly took our older daughter there, to a dude ranch outside the western entrance to Glacier National Park, but it was really to get my wife to the park. I went as a reluctant chauffeur between photographable vistas, only to fall in love with the place myself, especially the Swiss-like valleys formed by glaciers.

Still basking in the memories of a great trip, I then get a long epistle from Ted Maloney, who I'd know about more than known for years in the field. He works closely with Erica Swanson, the Montana Part C Coordinator, and they were interested in how I might be able to help them revamp the quality of their early intervention system. We quickly agreed that I would do some awareness workshops, plan with state leaders, and, with my colleague Amy Casey, conduct a Montana Routines-Based Interview bootcamp.

The workshops were productive. People from all over the state got to hear about the model, and I got to learn about how the state was organized and what the state of current practice was. Then we spent 2 days planning with state leaders--directors from the seven agencies responsible for Part C services along with one or more additioanal staff. Plus of course Erica and Ted. I facilitated the meeting, using the FINESSE II, an instrument for assessing typical and ideal practices. By the end of the first day, the group had decided on the seven practices they wanted to implement. The next day, we spent talking about implementation issues: barriers and solutions as well as a timeline. The seven practices they will work on implementing are

1. The Routines-Based Interview
2. Writing participation-based outcomes
3. Incremental (or additive) service decision making
4. Collaborative consultation to child care
5. Primary service provider
6. Consultative home visits
7. Consistent consultation by therapists

Oh, yes. I'm in love with Montana.

Early Intervention Contract Language

Many early intervention services are contracted for, so the contract should be a place for the contracting agency to outline expectations about how services should be provided under that contract. Yet most agencies do not take this opportunity and then feel besieged by problems with using contract services. The biggest gripe about contracted services is the lack of control over quality. I therefore encourage agencies to include practice requirements in their contracts. Here are some suggestions.

You really should push to end clinic-based services under the IFSP, but, because some states have not achieved that goal yet, the location clause could be dropped while maintaining the rest. I encourage agencies to include practice requirements, even when they think they have no choice but to contract with the only provider in their region. Just remember that the longer you let the seller dictate the market, the harder it will be to implement recommended practices. Many states have effectively stopped clinic-based services under Part C. They had to bite the bullet and pay only for services in natural environments. Whether you keep or discard the location clause, please make use of this language.

Practice Requirements

A requirement for fulfilling this contract is that practitioners will follow evidence-based family-centered practices, as outlined in the Agreed-Upon Mission and Key Principles for Providing Early Intervention Services in Natural Environments (Workgroup…; http://www.nectac.org/topics/families/families.asp) and as described by the practitioners’ professional organizations for pediatric practices for this age group. These practices are described here in terms of location, approach, teamwork, and compliance with the law.

Location

1.       Under this contract, practitioners will provide services in the natural environments, which are described in the law and regulations as the places where the child would be if he or she did not have a disability. These places are most commonly the family’s home or the child’s child care program.

Approach

Consultation With/Coaching Caregivers

2.       Practitioners will focus their work on using a consultative approach, also known as coaching, with the child’s caregivers to enhance those caregivers’ ability to provide interventions to the child between therapy sessions. This focus means practitioners

2.1.    Spend the whole session communicating with caregivers;

2.2.    Use collaborative not expert approaches in finding solutions (i.e., deciding on interventions); and

2.3.    Demonstrate strategies as necessary.

Focus on Functioning in Routines

3.       Practitioners will address outcomes on the individualized family service plan (IFSP), which will have target skills aimed at promoting functioning in routines. This means practitioners will promote

3.1.    Child engagement, including meaningful participation, in home, school, and community activities;

3.2.    Child independence, at the level the family wants, in routines; and

3.3.    Child social relationships, including communication and social-emotional skills, in routines.

Teamwork

4.       Practitioners will support the primary service provider (PSP) by

4.1.    Making joint home visits with the PSP (if the practitioner does not practice in natural environments—see Location above, welcoming the PSP on a visit to the clinic with the family);

4.2.    Expressing to the family confidence in the PSP’s ability to support them in carrying out the practitioner’s suggestions; and

4.3.    Providing information to the PSP.

Compliance With IDEA

5.       Practitioners will adhere to Part C of the Individuals with Disabilities Education Improvement Act, because this is the law under which early intervention services are funded. Even if a third party is paying for the practitioner’s service, if the practitioner is serving the child as part of early intervention (i.e., because the agency or practitioner is listed as a Part C service on the individual child’s IFSP), legally they must follow IDEA. The law stipulates that

5.1.    The IFSP team (not a doctor or another individual person) decides on what services are to be provided and at what frequency and intensity (i.e., prescriptions or practitioners’ recommendations on these matters do not dictate services);

5.2.    Services are provided to meet the needs identified in the IFSP outcomes (i.e., not based only on diagnosis or evaluation results for eligibility); and

5.3.    Changes in services, frequency, or intensity must be coordinated with the service coordinator and decided upon by the IFSP team (i.e., individual practitioners should not tell families how often they should see the child; they should discuss a potential change with the service coordinator first).

Failure to follow any one of these five practice guidelines can result in immediate termination of the contract. Practitioners can obtain more information from the contracting agency about this current approach to early intervention service delivery. 

The Future of Children: Children With Disabilities

We don't agree on what constitutes a disability, services are fragmented, and children in poverty might be missing out. The latest issue of The Future of Children, from Princeton and the Brookings Institution, is on children with disabilities. You can read the whole issue here. In their introduction, Janet M. Currie and Robert Kahn list five themes across the articles:

1. "It is remarkably difficulty to point to a consensus definition of disability." The one Halfon, Lrson, Newacheck, and Houtrow end up with is "an environmentally contextualized health-related limitation in a child's existing or emergent capacity to perform developmentally appropriate activities and participate, as desired, in society." Followers of this blog and the associated research and theory will notice (a) an emphasis on the role of the environment, which of course includes the humans in the child's ecology; (b) the notion that disability occurs when there is a limitation, not just a diagnosis (although I think "health-related" is a nod to the diagnosis); (c) the concept of goodness of fit between the demands of an activity and the abilities and interests of the child; and (d) the attention to participation, which is consistent with the ICF-CY and is closely related to engagement.
2. "The huge increase in recent years in the number of children who are considered to have a disability." And autism isn't even mentioned in this introductory article. "Researchers' efforts to track trends in disability and understand the meaning of the recent increase in numbers have been seriously complicated by changes over time in definitions of disability, in screening for disability, in services for disability, and in the extent to which particular conditions are considered to be actually disabling."
3. "The growing role of mental health issues in childhood disability." "Mental health disorders in childhood generally have larger impacts than childhood physical health problems in terms of adult health, years of schooling, participation in the labor force, marital status, and family income." In early intervention, many of us old-timers have perceived an increase in problems of self-regulation, attention, and engagement that become manifested as challenging behaviors. 
4. "Children live in families." Ever heard this before? The factors considered in this journal issue are (a) "Childhood disability poses major costs for families," (b) "the way that children are able to function within their families should be considered...,"  and (c) families often serve "as the only effective coordinators of care." We know about the loss of service coordination at age 3 and the challenges even before age 3. Much of the Future of Children emphasis is on economics, including public or private insurance, so the cost to families is discussed in some detail. In early intervention, we should think about " medical costs, indirect costs to families in terms of lost work time (especially for mothers), and costs in terms of losses to the child's future productivity. (They do not take into account the costs paid by private insurance or the cost of decreased well-being of families.)" From the health perspective that these articles are written, they tend to call service coordination "advocacy."
5. "The fragmentation of disability services." "A  disproportionate share of services for the disabled is still targeted at physical disabilities. Likewise, systems set up to deal with medical problems such as clinics for children with disabilities, or public insurance programs, are not coordinated with services at schools." Or with services in the community, such as homes or child care. The transition challenges we're familiar with show up in discussion of fragmentation even within the educational system: " For example, children receiving services for disabilities funded by special education before entering formal schooling are not automatically connected to special education services once in school." How services are paid for adds to the fragmentation, with children going in and out of insurance coverage. We know that in early intervention the fee-for-service approach to the therapies, using vendor therapists, has led to all sorts of problems with locations of services, how therapists think they have to work to get reimbursed, and the fact that multiple providers or agencies can be on one family's IFSP and rarely talk to each other.

The issue covers children of all ages, but it's helpful to have these economists and other experts, not necessarily from our own fields of early intervention or even special education, looking at the state of children with disabilities in America. They have suggestions for policy change. I am particularly interested in figuring out what individual early interventionists, program managers, and state administrators can do to inch us towards common sense and away from further problems. If you have ideas, please comment.