This month’s Carnival of Journalism topic, “What’s the best way — or ways — to measure journalism and how?” grew out of a post I wrote in February, Quantifying impact: A better metric for measuring journalism.
With this question, I wanted to broaden the possibile metrics beyond just impact. Higlights from the discussion are below. Enjoy!
Sheree Martin offered four important questions we must first consider:
- What is journalism?
- What is impact?
- How do we measure?
- Who is measuring?
Kathy E. Gill asked about similar fundamentals, “What matters? What is the role of journalism, our purpose, our challenge?” and raised a good question of balance:
how does “but it matters!” coexist in an environment where assessment is measured in large part by short-term page views and click-throughs?
Denise Cheng noted an opportunity to measure “inert engagement,” described as “the engagement that doesn’t want to come out of hiding as big steps like shares and comments.” She said:
measuring impact by measuring engagement are manifold. First and foremost, the modus operandi on my patch of the Internet is that journalism’s highest ideal is to equip its readership with information from which they can take judicious action
Specifically, she recommended defining metrics before you embark, measuring topical importance to the audience and acts accordingly, and that the sturdier your metrics become over time, the more of a road map you have.
Jonathan Groves looked at the issue on a more elemental level:
The root of journalism is truth, and the time-tested method that journalists have to uncover that truth is verification. If we want to measure journalism, it must begin here.
Addressing the quantitative vs. qualitative measurement distinction, he said:
Measurement assumes quantification, and some ideas — such as verification — are better evaluated qualitatively. Creating a measure requires including some attributes and excluding others; inevitably, such measures are always imperfect approximations, especially when it comes to complex concepts.
Clarisa Morales Roberts described a possible formula framed by the effects of journalism:
If we want to measure the impact of media and online journalism, we need to consider action. Action is what defines Effective Media (EM), and Effective Media can be measured by the Action that is a direct result of Quality Dialogue that is Shared
…
So, if we want to consider Impact by measuring Action, that measurement has to be proportional.
Michael Rosenblum emphasized the importance of finding niche instead of mass audiences:
As more and more content begins to fill the blogosphere and cyberspace and the cloud and wherever else ‘it’ all is, the competition for the Holy Grail of mass audience becomes ever more intense, and as such, the content itself becomes ever more amorphous.
Yet where is the ‘real’ value?
The web gives us access to discrete groups with specific interests. Our goal should be ‘narrowing the field’, not expanding it. Creating affinity groups with a common interest and common goals, and then, making it possible for those people to achieve those goal – whether its contributing to a new project – as in Kickstarter, or going on a golfing trip to St. Andrews.
Steve Outing looked at how social media is gaining an edge in the impact realm:
When I look at the question, I can’t help but get sidetracked into thinking how social media (i.e., “the crowd” utilizing digital social tools like Twitter, Facebook, and Change.org, among others, to amplify their voices) in a growing number of cases is having more impact than the traditional news media can achieve themselves — or is driving the mainstream news media to pay attention to stories that their editors fail to recognize as important.
Carnival ringmaster David Cohn also proposed an alternative approach :
I want to measure a different kind of impact. The impact of the dollars we spend in pursuit of journalism and its meaningful impact.
…
What we don’t appreciate is the strength of the little guy. What they don’t have in “impact” they do have in efficiency.
Steve Fox challenged the assumption of measuring something like impact:
Perhaps we all need to remember that the true “impact of journalism” rests with the impact we have on people’s lives. Have we given readers/viewers an amazing piece of writing or video that makes them appreciate parts of their life more? Have we created an “Oh, wow” moment for readers/viewer? Have we expanded someone’s universe? Isn’t that why we got into this business? Isn’t that what journalism has always done?
Perhaps the real question should be: “Why are we spending so much time measuring the “impact” of journalism?” Because, it really isn’t quantifiable now, is it?
What would be your ideal measure for journalism?
I agree with Jonathan Groves that measuring journalism starts with verification, and I think this is at the core of journalism ethics. But the place of journalists in the larger network flow of information creates new ethical possibilities and challenges. A new journal article by Alfred Hermida (http://bit.ly/I6YoME) argues that the developing potential of collective verification through social media is challenging the traditional role of journalists. I think we have to keep a central place for journalists’ own work of verification and, however difficult, their efforts to establish truthfulness of information. But I think it’s also important to acknowledge that their work every day sits within a network capable of doing important work of verification–while still recognizing the potential for harm through error.