These quotes from an interview Sue Gong did of me identify some of my fundamental purposes for my research and this blog. The interview was part of a study she did of my evaluation life as one of my students. Her full report will be available as one of the documents here in the folder of interviews of people who chose to be identified.
Interview: SPG with DDW
7/10/13
S: Are there any viewpoints or issues you think that I ought to know?
DDW: I’m wondering as you look through all that stuff that I’ve said. A big fear that I have is that I am on a big fishing expedition to support my own biases. In some ways I am doing that, but I also don’t think that’s what I should be doing. I should be discovering what these people are feeling and thinking and not just confirming what I am thinking and feeling because I’m sure they don’t. Do you feel that Williams is looking for certain things and finding them in Stake, Stufflebeam and Scriven? Am I missing things? Am I imposing my ideas on them?
S: We can’t sidestep the fact that some things are more interesting to us than others, and that we have ideas about what is real. That is the real challenge of doing this. But I think in your questions you have tried to be very open about where people are leading. I think you have identified some seminal questions, so they really respond and give you very interesting and very different stories. If your process were controlling you wouldn’t get that diversity that you are getting. One of your questions that is very interesting is one that they find interesting too-- your question about the rationality continuum. . .
As I go over this interview I am struck by how DDW keeps turning it around. He is asking me questions. And I keep responding. And that doesn’t count the part of the interview that I deleted in which we spend 15 minutes discussing my AEA Ignite project and my views on the thematic matrices Jacque and I are producing about the evaluators.
DDW: In 2012 at AEA it is mainly telling their stories and not commenting on them. In 2013 it is me commenting on them and getting their response to my comments. One of the big things is that people have informal evaluation lives and formal evaluation lives and that there is a connection between them. And that informal evaluation life influences their formal evaluation life. I don’t know if that comes through, or if it is in the background, or if it is me leading them into it.
S: Well, everyone is responding with these powerful stories and responses. So I think that you have identified some core questions so people are saying yes.
DDW: Well it’s not this huge fear. I can leave it up to somebody else to point out if I have done that. I can deal with it when they do that. I’ve been getting ready to go to Colorado Monday to visit my son and daughter-in-law. They’re about ready to have their baby. He’s been doing Teach for America for five weeks in Tulsa, OK. They were hoping the baby wouldn’t be born until they were through. So we are going to see the baby and go on to Indiana to visit our son there. And then we’ll come back for a day or two. And then go to Arizona to a condo for a week with my parents. Then after a week we’ll be in California for another family reunion. I just had another interview with Jennifer Greene on Monday and realized that I want to get started on things for that bigger book so that I can have a more targeted set of things to ask people. I have a bunch of interviews to do with people who have told me yes, that they would like to be interviewed. Jody Fitzpatrick, AEA President, told me last year “Sure, interview me.” I’ve been putting it off until I have a more targeted goal.
S: What would you teach people about evaluation?
DDW: In our ward’s girls’ camp I went up with this fellow in the ward who is about 52. His oldest child is about the same age as my second youngest. We’ve interacted with each other for years. I decided that if we were going to be driving for1 and 1/2 hours, I asked him if I could interview him. This is the topic we concluded on because we were standing around the fire last night and he asked me “Why are you doing this? What are you interested in?” I basically told him that the thing I’m most interested in is overcoming the biases that I see in professional evaluation writing that says “People are terrible evaluators and they need to pay professionals to do it.” So I acknowledge that most of us are pretty bad evaluators. There is this research that Kahneman and others have done that show that people are bad evaluators and are inconsistent. And it’s true that people are inconsistent and I don’t know that that can be overcome by education. Nobody could afford to have professionals do it all for them. We make so many judgments every day. Yeah. I agree the natural man doesn’t want to take responsibility to evaluate, especially hard things in their lives, that they might have to change. But that is our responsibility to do it.
The other side of the argument is that nobody can do it for us. Evaluations are lived out moment by moment and nobody can live our lives for us. Even if somebody tells us, “I would advise you do this,” we would have to evaluate it. We have the responsibility and we cannot divest ourselves of it. And there are so many of them that we couldn’t pay somebody to do them for us even if we had millions of dollars to pay them. So it seems to be the ultimate point is: We are here to learn how to evaluate. We ought to teach them to do it better, and not do it for them. Everything in the Gospel points to that. You teach a man correct principles. You teach him responsibility. You teach a man to fish. You don’t give him fish all the time. It’s an Agency responsibility. It’s a welfare principle. Its a Stewardship principle. And so the only solution is we have got to educate people. Your question is how would I do that?
S: What would you teach?
DDW: The first thing I would teach them is what I teach in my classes. The first thing I would teach them is to see, “ Oh I am an evaluator. That’s part of what I do.” Then teach them to metaevaluate their own evaluations and say, “Oh I am not very good at this part or that part. I need to work on that.” I would invite them to go through that cycle a few times and do the same thing with myself so that I am illustrating it for them. I would demonstrate. It would be nice if they would share with others in some form. So that they could see and say, “ Oh I see you are metaevaluating and asking these questions. I’ll ask some of them about myself.”
Last week I interviewed another guy that is going to China to teach English. He’s been studying up on Chinese culture and language and so on. And he talks about something that I have heard my Chinese students say over the years that Mao made this big deal about self criticism. I’m sure you’re familiar with that. Right?
S: Uh huh.
DDW: And so what I’m talking about isn’t that, I don’t think. The point isn’t to look for bad things in yourself and tell others, “I’m terrible. So I’m one of the masses, and I should be beaten down” or anything. The point is that we should be reflective in thinking about “How well am I evaluating and how can I evaluate better and how could I help the people in my stewardship evaluate better by being self-reflective and metaevaluative.”
So mostly what I would teach is: You are an evaluator. You are not the best evaluator you could be. What are some things you could work on to improve? And just make that a part of life.
And that’s really the same thing I do with clients as a formal evaluator. This thing that Randy Davies and I have been doing for the last couple of years with the State of Utah. They’ve asked us to go ahead and work with them for another year now. And we’re saying “Let us help you think through how to make evaluation and metaevaluation part of your program among the higher ed faculty of programs in these four discipline areas. So it’s just inviting people to be reflective and metaevaluative. I’m interested in doing that in several ways. That’s where the books that I want to write come into it.
So that is what I was doing last night with this guy who was an attorney and is now teaching legal writing for fifteen years. The question I put to him is, “Is the evaluation system or process taught in law school a good one? I’m interested in how the people I go to in different disciplines evaluate within their profession. But I’m also interested in having parents metaevaluate, “How am I doing as a mother or father?” Or kids or grandparents. And I’d love to do the same with church leaders and community leaders. Anybody.
The reason I’m so interested in the educative dimension is that to me the essence of what it means to be human is we can evaluate the difference between good and evil. If the statement that professional evaluators make is true, then people are not very good at doing that. So I would like to understand how people do it first, and then help them think about how they could do it better.
Not that I would come in and tell them how to do it better. I don’t really have that in mind. But I could give them somethings to think about like “Have you taken into account what other stakeholders’ values are? Do you realize what values you are putting forward here? Are those really your most important values? How are you defining the evaluand? Is it really you and not others? Is it the program? Or is it some combination of those things? Those are the kinds of questions I would like to ask people as kind of their evaluation tutor.
This guy I was telling you about that I interviewed last week, I interviewed two guys who wanted to be life coaches. One of them is a life coach and the other one wants to do it when he gets back from China. In a way they are doing what I’d like to do. They are meeting with people and asking the same kinds of questions I’m asking. They don’t call it evaluation. But they are asking people, “So what do you want to do with your life? What do you think about what you are doing? What are you going to do about the fact that you don’t like how you are doing? What are you going to do about it? Essentially they are inviting them to evaluate their own lives?
Comments
Post a Comment