Toward Valuing With Critical Systems Heuristics
Abstract
Evaluation is defined by its central task of valuing—the process and product of judging the merit, worth, or significance of a policy or program. However, there are no clear-cut ways to consider values and render value judgments in evaluation practice. There remains contention in the evaluation field about whether and how to make value judgments. No approach to valuing eliminates the uncertainty, plurality, and potential for conflict that comes with considering values. This article explores what critical systems heuristics (CSH), an area of applied systems thinking, might contribute to four long-standing issues regarding valuing: envisioning the social value of evaluation, framing the evaluand and evaluation, selecting and justifying criteria, and determining the roles of the evaluator(s) and stakeholders in valuing. CSH contributes concepts and tools that, in theory, support more reflective, responsible valuing although further practical application is needed.
Get full access to this article
View all access and purchase options for this article.
References
Alkin M. C., Vo A. T., Christie C. A. (2012). The evaluator’s role in valuing: Who and with whom. New Directions for Evaluation, 133, 29–41.
American Evaluation Association. (2004). American evaluation association guiding principles for evaluators. Retrieved from http://www.eval.org/p/cm/ld/fid=51
American Evaluation Association. (2011). American evaluation association public statement on cultural competence in evaluation. Fairhaven, MA. Retrieved from www.eval.org
Boyd A., Geerling T., Gregory W. J., Kagan C., Midgley G., Murray P., Walsh M. P. (2007). Systemic evaluation: A participative, multi-method approach. Journal of the Operational Research Society, 58, 1306–1320.
Chelimsky E. (2012). Valuing, evaluation methods, and the politicization of the evaluation process. New Directions for Evaluation, 133, 77–84.
Chelimsky E. (2014). Public-interest values and program sustainability: Some implications for evaluation practice. American Journal of Evaluation, 35, 527–542.
Churchman C. W. (1968). The systems approach. New York, NY: Basic Books.
Churchman C. W. (1971). The design of inquiring systems: Basic concepts of systems and organizations. New York, NY: Basic Books.
Churchman C. W. (1979). The systems approach and its enemies. New York, NY: Basic Books.
Datta L. E. (2011). Politics and evaluation: More than methodology. American Journal of Evaluation, 32, 273–294.
Davidson J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: Sage.
Fetterman D. M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Fetterman D. M., Wandersman A. (2005). Empowerment evaluation: Principles in practice. New York, NY: The Guilford Press.
Greene J. C. (1997). Evaluation as advocacy. American Journal of Evaluation, 18, 25–35.
Greene J. C. (2006). Making the world a better place through evaluation. In Alkin M. C. (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (pp. 208–217). Thousand Oaks, CA: Sage.
Greene J. C. (2011). Values-engaged evaluations. In Segone M. (Ed.), Evaluation for Equitable Development Results (pp. 192–206). United Nation’s International Children’s Emergency Fund.
Greene J. C., Destefano L., Burgon H., Hall J. (2006). An educative, values-engaged approach to evaluating stem educational programs. New Directions for Evaluation, 109, 53–71.
Gregory A. J., Jackson M. C. (1992a). Evaluating organizations: A systems and contingency approach. Systems Practice, 5, 37–60.
Gregory A. J., Jackson M. C. (1992b). Evaluation methodologies: A system for use. Journal of the Operational Society, 43, 19–28.
Gregory A. (1997). Evaluation practice and the tricky issue of coercive contexts. Systems Practice, 10, 589–609.
Hall J. N., Ahn J., Greene J. C. (2011). Values engagement in evaluation: Ideas, illustrations, and implications. American Journal of Evaluation, 33, 195–207.
Henry G. T. (2002). Choosing criteria to judge program success: A values inquiry. Evaluation, 8, 182–204.
Hood S., Hopson R., Kirkhart K. (2015). Culturally responsive evaluation: Theory, practice, and future implications. In Newcomer K., Hatry H., Wholey J. (Eds.), Handbook of practical program evaluation. San Francisco, CA: Jossey-Bass.
House E. R. (2001). Unfinished business: Causes and values. American Journal of Evaluation, 22, 309–315.
House E., Howe K. (1999). Values in evaluation and social research. Thousand Oaks, CA: Sage.
Hummelbrunner R., Reynolds M. (2013, 6). Systems thinking, learning and values in evaluation. Evaluation Connections: The European Evaluation Society Newsletter, pp. 9–10.
Julnes G. (2012a). Managing valuation. New Directions for Evaluation, 133, 3–15.
Julnes G. (2012b). Developing policies to support valuing in the public interest. New Directions for Evaluation, 133, 109–129.
Kant I. (author), & Smith N. K. (translator) (1929). Immanuel Kant’s critique of pure reason. Boston, MA: Bedford.
Levin-Rozalis M. (2014). Let’s talk program evaluation in theory and practice. Tel Aviv, Israel: Dekel Academic Publishing House.
Mabry L. (2010). Critical social theory evaluation: Slaying the dragon. In Freeman M. (Ed.), Critical social theory and evaluation practice. New Directions for Evaluation, 127, 83–98.
Mark M. M., Henry G. T., Julnes G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving public and nonprofit policies and programs. San Francisco, CA: Jossey-Bass.
Mertens D. M. (2006). Social transformation and evaluation. In Alkin M. C. (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences. Thousand Oaks, CA: Sage.
Midgley G. (1992). The sacred and profane in critical systems thinking. Systems Practice, 5, 5–16.
Midgley G. (1996). Evaluating services for people with disabilities: A critical systems perspective. Evaluation, 2, 67–84.
Midgley G. (2000). Systemic intervention: Philosophy, methodology, and practice. New York, NY: Kluwer Academic/Plenum.
Midgley G. (2007). Systems thinking for evaluation. In Williams B., Imam I. (Eds.), Systems concepts in evaluation: An expert anthology (pp. 11–34). Point Reyes, CA: EdgePress of Inverness.
Morris M. (2012). Valuation and the American evaluation association: Helping 100 flowers bloom, or at least be understood? New Directions for Evaluation, 133, 85–90.
Patton M. Q. (2012). Contextual pragmatics of valuing. Promoting valuation in the public interest: Informing policies for judging value in evaluation. New Directions for Evaluation, 133, 97–108.
Reynolds M. (2007). Evaluation based on critical systems heuristics. In Williams B., Imam I. (Eds.), Systems concepts in evaluation: An expert anthology (pp. 101–122). Point Reyes, CA: EdgePress of Inverness.
Reynolds M. (2010). Evaluation and stakeholding development. The 9th European Evaluation Society International Conference, Prague, Czech Republic.
Reynolds M. (2014). Equity-focused developmental evaluation using critical systems thinking. Evaluation, 20, 75–95.
Reynolds M. (2015a). (Breaking) The Iron Triangle of evaluation. IDS Bulletin, 46.
Reynolds M. (2015b). Rigour (-mortis) in evaluation. Evaluation Connections, pp. 2–4.
Reyonlds M., Williams B. (2011). Systems thinking and equity-focused evaluation. In Segone M., Bamberger M. (Eds.), Evaluation for equitable development results (pp. 115–141). New York, NY: UNICEF.
Schwandt T. A. (1997). The landscape of values in evaluation: Charted terrain and unexplored territory. New Directions for Evaluation, 3, 25–39.
Schwandt T. A. (2003). In search of the political morality of evaluation practice. Studies in Educational Policy and Educational Philosophy, 2, 1–6.
Schwandt T. A. (2005). Value judgment & values. In Mathison S. (Ed.), Encyclopedia of evaluation. Thousand Oaks, CA: Sage.
Schwandt T. A. (2008). Educating for intelligent belief in evaluation. American Journal of Evaluation, 29, 139–150.
Schwandt T. A. (2015). Evaluation foundations revisited: Cultivating a life of the mind for practice. Stanford, CA: Stanford University Press.
Schwandt T. A., Gates. (2016). What can evaluation do? An agenda for evaluation in service of an equitable society. In Donaldson S. I., Picciotto R. (Eds.), Evaluation for an equitable society. Charlotte, NC: Information Age.
Scriven M. (1991). Evaluation thesaurus. Newbury Park, CA: Sage.
Scriven M. (2006). Conceptual revolutions in evaluation: Past, present, and future. In Alkin M. C. (Ed.), Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences. Thousand Oaks, CA: Sage.
Scriven M. (2012). The logic of valuing. New Directions for Evaluation, 133, 17–28.
Shadish W. R., Cook T. D., Leviton L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
Shipman S. (2012). The role of context in valuing federal programs. New Directions for Evaluation, 13C3, 53–63.
Stake R. E. (2004). Standards-based and responsive evaluation. Thousand Oaks, CA: Sage.
Stake R., Migotsky C., Davis R., Cisneros E. J., Depaul G., Dunbar C.…Chaves I. (1997). The evolving syntheses of program value. Evaluation Practice, 18, 89–103.
Stake R. E., Schwandt T. A. (2006). On discerning quality in evaluation. In Shaw I. F., Greene J. C., Mark M. M. (Eds.), Handbook of evaluation (pp. 404–418). London, England: Sage.
Ulrich W. (1983). Critical heuristics of social planning: A new approach to practical philosophy. Switzerland: Swiss National Science Foundation.
Ulrich W. (1987). Critical heuristics of social systems design. European Journal of Operational Research, 31, 276–283.
Ulrich W. (1988). Churchman’s “Process of unfolding” and its significance for policy analysis and evaluation. Systems Practice, 1, 415–428.
Ulrich W. (2000). Reflective practice in the civil society: The contribution of critically systemic thinking. Reflective Practice, 1, 247–268.
Ulrich W. (2002a). Boundary critique. In Daellenbach H. G., Flood R. L. (Eds.), The informed student guide to management science. London: Thomson Learning.
Ulrich W. (2002b). Critical systems heuristics. In Daellenbach H. G., Flood R. L. (Eds.), The informed student guide to management science (p. 72). London: Thomson Learning.
Ulrich W. (2005). A brief introduction to critical systems heuristics (CSH). Retrieved from http://wulrich.com/downloads.html
Ulrich W. (2011). What is good professional practice? Part 1. Werner Ulrich’s Webpage. Retrieved from http://wulrich.com/downloads.html
Ulrich W. (2012). CST’s two ways: A concise account of critical systems thinking. Retrieved from http://wulrich.com/downloads.html
Ulrich W. (2014). A primer to critical systems heuristics for action researchers. Hull, UK: University of Hull, Centre for Systems Studies. Retrieved from http://wulrich.com/downloads.html (originally published in 1996).
Ulrich W., Reynolds M. (2010). Critical systems heuristics. In Reynolds M., Holwell S. (Eds.), Systems approaches to managing change: A practical guide (pp. 243–292). Milton Keynes, England: Springer.
Williams B. (2015). Prosaic or profound? The adoption of systems ideas by impact evaluation. IDS Bulletin, 46, 7–16.
Yarbrough D. B., Shulha L. M., Hopson R. K., Caruthers F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users. Thousand Oaks, CA: Sage.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: July 21, 2017
Issue published: June 2018
Keywords
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in American Journal of Evaluation.
View All Journal MetricsArticle usage*
Total views and downloads: 1591
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 22 view articles Opens in new tab
Crossref: 35
- Digital health interventions for adolescents living with HIV in low- and middle-income countries: A narrative review and logic model
- Designs for Language Program Evaluation
- Critical Systems Thinking
- ‘Good’ evaluation: Methodological diversity may be empress, but sound methods remain queen
- Rich Pictures: A Visual Method for Sensemaking Amid Complexity
- Critical Systems Heuristics: a Systematic Review
- Exploring Perspectives on Agroecological Transition in Scotland with Critical Systems Heuristics
- Equity “On the Sideline”
- Didactic Features Specific to Green Chemistry Teaching in the Journal of Chemical Education
- Exploring Perspectives on Agroecological Transition in Scotland with Critical Systems Heuristics
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
AEA members can access this journal content using society membership credentials.
AEA members can access this journal content using society membership credentials.
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
