The WAA Championship is an annual activity where members are invited to review the association’s site and traffic data and put their analyst skills to work. After some hiccups with the access to data that delayed the starting date, the analysts analyzed, submitted their reports, insights and observations about the WAA site. The judges deliberated and had to take difficult decisions. Below we present the criteria used to judge the submitted papers and, of course, the winners.

Competition Criteria

There were four criteria against which the analyses were judged:

1. Business Goals: Were they clearly identified? (25 pts)
   Macro goal: Membership
   Micro goal: Member value, Recency, Frequency, Engagement
   Note: It was OK to not identify the macro and micro goals, but it was important to clearly identify some business goals.
2. Analysis: Was there enough data? (25 pts)
   Segmentation to provide context
   Correlations or other demonstration of relationships
   Multiple data sources used (more than just the web analytics tool)
3. Recommendations: Were the insights valuable? (25 pts)
   Meet SMART criteria (Specific, Measurable, Actionable, Realistic, and Time Bound)
   Informed by data
4. Overall quality (25 pts)
   Is the analysis structured, succinct, and focused?
   Appropriate use of visuals and data (graphs, charts, etc.)
   Writing and presentation quality

Fatal Flaws

After analyzing the entries, the judges agreed that some mistakes were so significant that if made hurt the credibility of the analysis, and in turn, would make it unlikely that the recommendations would be acted upon. Entries that contained a “fatal flaw” were eliminated from consideration. These “fatal flaws” were:

• Unfocused Analysis: When the focus of the analysis is not tied to business or marketing objective(s), the analysis is at best interesting trivia, but not important insight.
• No Compelling Story: The judges were unable to come to consensus on what it means to “use the data to tell a story.” However, they easily reached consensus about which entries had this fatal flaw. In some cases, the analyses with this fatal flaw were written stream of conscience and difficult to follow. In other cases, analyses with this fatal flaw were so boring that they failed to instill a sense of relevance, importance or urgency.
• Factual Errors: One bad apple spoils the bunch. One data error calls the rest of the data into question (or more importantly, the recommendations based on that data). It was particularly easy to spot factual errors when the analyses were mostly based on the same data sets.
• Unfounded Recommendations: The judges agreed that it is easy to report data and it is easy to make recommendations. However, limiting your recommendations to those based on the data is harder than it looks, especially when all the data that you want isn’t available. The result is unfounded recommendations that undermine a data-driven culture. Many of the unfounded recommendations in the analyses were usability related, such as recommendations to shorten the registration form. This would be a valuable recommendation if it were based on data that indicated that the length of the form was the key obstacle to completion. Or, even if there was data to indicate the form is a key obstacle in the member registration process. In fact, that data might exist, but it wasn’t referenced in the analysis.
• Inappropriate Tone: Entries were given much leeway, but the judges agreed that it was inappropriate for an analysis for this competition to be condescending or whimsical.
• Excessive Typographic Errors: Having one typographical error is not a fatal flaw, but an abundance of typographic errors demonstrates inattention to detail and makes it difficult to trust that the recommendations were based on good data. Blatant typos in the file name became a particular pet peeve, but didn’t eliminate anyone from consideration.

The Winners

Unfortunately, the majority of the entries didn’t make the cut because they contained one or more fatal flaws. Of those who met the criteria, there was a wide margin between the top two finalists and the rest of the entries. So, rather than select five finalists, the judges decided to only select two. Despite the focus on data, the winning entries were outstanding because of feelings they inspired. The winning entries were compelling and motivating. The judges immediately wanted to share the analyses with the WAA board members and discuss how to begin implementing the recommendations.

You can read the winning analyses here:

1. First Place: Submitted by Joel Wright and Ed Wu, both from the Dell Online Analytics Team
2. Second Place: Submitted by Leo Burnett team

Of course, if you’re interested in helping the WAA implement any of these recommandations, please leave a comment in this post or contact us by email.

The Judges & Team

The WAA is a volunteer-based organization and we want to thank the following:

• Judges: Jennifer Veesenmeyer, Judah Phillips, Stéphane Hamel and "Wandering" Dave Rhee.
• Logistic & Organization: Daniel Waisberg, June Dershewitz, Clare Madden, Shannon Taylor, Suzanne Carter-Jackson.

Comments

[Ed]

Thanks Daniel and all who volunteered to make this a success!

I'll be interested in working with the board, the new ED and any volunteers to implement these recommendations going forward.

With the new website just launched, we have got a much more robust technology but the need to focus on and improve member value doesn't go away and we know we have lots of works to do!

cheers,
Ed

[Jonghee Jo]

Congratulations Ed! Very nice work. You are the real "Super Web Analyst"!

Jonghee Jo

[Joel Wright]

Thank you, WAA!

Ed and I are excited to have participated in this contest. Thank you, WAA, for holding this every year.

Congratulations on launching the new site by the way.

I am interested in helping improve the member value of the WAA website as well.

Thanks!
Joel

[Michael]

I have to admit that I like 2nd place much more than 1st place. 2nd place winner split the analysis into bite size format, and present an Observation and an opportunity for each objective. For someone like me who easily gets bored from reading essay, I love 2nd place's paper.