Previously Unpublished: Analyzing Eric's Calculation or “Back into the fray, picking up at Eric's 'The purported complexity of my calculation…'”

by Joseph Carrabis on July 1st, 2009

I decided to continue the conversation started on Starting the discussion: Attention, Engagement, Authority, Influence, … in a new post for several reasons. People wanting to know those reasons can find them over on Problem with Blogs for Holmes…for this Holmes, anyway. I'll be posting some “required readings” to what I write here over on BizMediaScience in an attempt to make my posts shorter here.

Caveat: Some first readers of this post were concerned that people unfamiliar with my methods may interpret what I write here as an attack on Eric or his calculation. This is not my intent nor is it the case. My last statement (or close to) in this post is “Would anyone care to join me in helping Eric reformulate this pseudo-logic so that it does something closer to what is intended?” and that offer is sincere. Companies, scientific societies, journals, etc., regularly hire me to evaluate their material prior to publication and I went at this analysis with that same mindset. One first reader wanted to know if anyone had asked me to do this analysis and no, not directly. Eric responded to one of my comments with “The purported complexity of my calculation…” and thus entered his calculation into the discussion. To understand his points I wanted to understand his calculation. Simple as that.

I left off on Starting the discussion: … with “Regarding your equation…hmm…

Give me a bit of time to read through your statements before I offer anything on “The purported complexity of my calculation accounts for that.”

Starting from there…

The first thing I'll note is that this is a definition of “engagement”, an excellent note as it narrows the analysis greatly because now all that's left is to determine

1) if the definition has merit and

2) if the calculation correctly mathematizes the definition.

Completely different problems to solve are

“How many definitions are out there?”

“Are any of these other definitions better at demonstrating ROI or some such?”

“Do any of these other definitions more closely model a reality people care about?”

That list can be interestingly long.

And as always, a NextStageish question, has anybody looked at the calculation itself and determined if it makes mathematical and arithmetic sense? I did see that Frank Flaubert, Gary Angel, Nick Arnett and others had commented on what kind of data is collected and from where but I didn't see anyone actually looking at the calculation itself, its actual functionality, to determine if it would calculate something that stood up to simple analysis. I'm sure someone has and sorry to cover the same territory. Let this be an example that I should have spent more time studying the situation than responding.

I kept in mind key phrases such as “…new ways to examine and evaluate visitor interaction.” , “…time-consuming…”, “…identify truly qualified opportunities.”, “…this model fully supports both quantitative and qualitative data,” and “…there is no single calculation of engagement useful for all sites,” during the analysis.

The analysis itself can be found on An Analysis of Eric Peterson's “Engagement” Calculation. It's about 8 pages long, went through 3 edits and was reviewed by two other researchers. What follows here is the result.

There is a section of How to measure visitor engagement, redux entitled “How Does This All Work in Practice?” Let me first offer that the economic value of a metric is directly proportional to

1) the information value of what the metric reports on

2) as that information value is defined by some group with an interest in it and

3) that same group's ability to change environmental factors so that

4) the metric changes report value (not information value) in direct and obvious response to that same group's intentional changes in environmental factors.

Thus the proposition (and using the same lexical substitution as noted in An Analysis of Eric Peterson's “Engagement” Calculation (lexical substitution explained)) “…someone coming from a Google search for web analytics demystified who looks at 10 pages over the course of 7 minutes, downloads a white paper and then returns to my site the next day will have a higher visitor [degree and depth of visitor interaction on the site against a clearly defined set of goals] value than someone coming from a blog post who looks at 2 pages and leaves 2 minutes later, never to return.” is obviously true for items 1 and 2 above (and with the caveats and concerns already documented on An Analysis of Eric Peterson's “Engagement” Calculation) and I have no evidence regarding items 3 and 4.

Does it have anything to do with the dictionary definition of engagement? Not that I can determine.

So, if the final offering is that some pseudo-logic has been created and the term “engagement” has been used as a symbol for that pseudo-logic then great and good. But the value of this semanticism will only go as far as a business case can be made according to the four items above, me thinks. And it's not “engagement” by any definition other than the one supplied in How to measure visitor engagement, redux.

Final notes: I went through this exercise because and as suggested in the original premise of TheFutureOf, there is more power in different disciplines coming together than in any single discipline attempting to answer what it doesn't have the tools to question.

So again to state what I wrote at the beginning, Would anyone care to join me in helping Eric reformulate this pseudo-logic so that it does something closer to what is intended? I'm game for it. Remember, though, it might take me a while to respond.