My reflections on the recent Australian Evaluation Society conference

I got my evaluation break working at an evaluation consulting company in 2005. (Year may not actually be correct, it was a long time ago). When I started this job, the office manager said to me, ‘you know what, Lauren, you remind me of one of the cows on my dad’s farm… In the morning, all the cows would line up to be milked no problem. There was this one cow would never cooperate and make lots of noise, and it was a huge headache to get her into the milking shed.’ Many years later, it occurred to me that why should the cow cooperate in giving you milk when she gets nothing in return? ‘Good for that cow’, I thought.

Anyway, the point of that story is that I am naturally contrarian and obstinate, and you should keep that in mind when considering my reflections. I also looked at many of the past AES evaluations and can see I might be in the minority with some of my views. Something to consider. :)

Here is a list of my reflections in no particular order. I hasten to add that many of my suggestions would involve significant reorientation of how the conference is funded and organised. I know some of these suggestions may not work practically. Still, I present this list for reflection.

1.       My first AES conference was around 2007. Since then, I have not seen a meaningful difference in types of presentations, topics, methods, or themes. There is a strong focus on social programs with a smaller number of presentations on international programs and some natural resource management representation. I see the same speakers every year.

2.       There does not appear to be a wide range of ideological diversity in the types of presentations. Where is the opportunity to be challenged by someone with a completely different mindset and approach to doing things? It feels like certain types of sectors and ideologies are self-selecting out of participation.

3.       There should be a panel of First Nations people who review all First Nations submissions and control a conference stream that focuses on First Nations matters. If a presentation is about First Nations evaluation and does not have a First Nations person on the speaking panel, it should be automatically barred.

4.       There is not enough investment in seeking the perspectives of people/organisations who participate in evaluation as ‘the evaluated’ - this could include any type of participant such as government, program staff, young people, people with disability, people from culturally diverse backgrounds, and so on. They are often only included when they had a good experience and are sought out by the evaluator to attend.

5.       There is not enough investment in seeking the perspectives of organisations that have undergone evaluation. They seem to be most likely to get represented when they had a good experience of an evaluation, and then the evaluator asks them to come to the conference.

6.       I would like to see a stream for people who disliked being part of a process of evaluation, or various parts of evaluation, to come and share their experiences. I would like to see a grant offered for people who did not have a good experience of evaluation to come and share their experience at the conference.

7.       The entire review of conference submissions should be blind, not just the first selection round. Having said that, relying on submissions for conference content appears to miss opportunities for diverse and divergent voices and should only constitute one part of curating content.

8.       The process of seeking submissions should be supplemented with an ongoing approach to identifying people with an interesting story to tell about evaluation and then seeking their participation in the conference.

9.       The panel/audience approach for sharing information is not effective anymore in a time we can access any information we want on the internet. We can watch lectures online. At the same time, there is enormous value in having everyone in the room face-to-face. There needs to be a new approach to sharing and exchanging information.

10.   The Indigenous conference grants are not administered well. The per diem is not enough for most delegates. Grant recipients coming in from the Pacific are not provided with enough support. Grantees don’t appear to be well supported during the conference. And, for the love of god, please stop asking grant recipients to stand up on stage during the gala dinner. They earnt their place there just like the rest of us and if you want to bring them on stage, give them an opportunity to share their stories and experiences.

11.   I think Western middle-class society in general needs to consider what it is offering to the field of evaluation today. Regarding social sciences, our ways of doing, being, and knowing feel like they are stagnant. They feel like they have been stagnant for a long time. How can we stop being the lead custodians and power holders of this work, and meaningfully hand it over to others who can breathe some new vitality and life into this field?