Originally posted by Jim Novo.

The WAA has published a lot of info about the new WAA Certification Exam; you might want to first read the FAQ and take a look at the application information and Exam Handbook for the organizational details, and you can see sample questions from the Test. But something I can just about guarantee about the Certification – no matter how much info the WAA publishes about it, many people will still have questions!

So here, I will attempt to answer other kinds of questions I think people might have based on my discussions with WAA members.

However, I’m going to approach this topic a bit differently than most of the published documentation – from a Product / Marketing perspective, rather than an Educational / WAA POV. I can do this because (if you don’t know) I have worn all the hats on this project – developer, marketer, WAA project owner – and I think it might be helpful to tell the business story of the WAA Certification, from the bottom up.

And if you have other questions, feel free to leave them in Comments and I will do my best to answer them!

Where did the idea for Certification come from?

The WAA is a member-driven organization; we listen to the membership and try to accomplish what they would like us to accomplish. We heard from hiring folks and managers that ”web analysts today know a lot of the buzz words and can follow instructions as far as reporting goes, but what we’d be willing to pay a premium for is web analysts who discover things on their own, who add value in areas we don’t already know about”.

So that’s where WAA Certification came from. It addresses a specific need identified by members, what came to be known internally as the “Book Smart versus Sherlock Holmes” problem. Sure, you can read a ton of books or blogs and be a good web analyst by following best practices. But so can a lot of other people. What you need to pass the Certification Test is different; you have to be able to turn data into insight and recommend a best action given the scenario presented.

How come the WAA’s Educational efforts lack “tool focus”?

Because the tool vendors own that focus, and by definition they have the resources to be much better at tool education / certification than the WAA, so why would be want to compete with the tool vendors?

Better to add value on the business side, where there is demand we can fill and a lack of trusted resources. And if you think about it, this approach simply expands the overall WA opportunity. People who want to become experts on the tool side have a path (through the vendors), and people who want to become experts on the analysis / business side also have a path through the WAA. And if you want to be a Universal Web Analytics Soldier, I guess you could do both!

Does that mean I can pass the Test with No Tool Knowledge?

Not at all. The threshold we set is you need to be able to communicate effectively with tool experts to pass the test. That means you will need to know the basics of how the web works, how the tools accomplish their mission, and know what all the web analytics terms mean. Example: To pass the Test, you don’t need to know how to write a tag, but you do need to know when a custom tag is required and how to communicate your need effectively.

So Marketing people can Pass the Certification Test? eCommerce Managers? Usability people? Media Buyers? Etc.?

Absolutely, if they are good at transforming the data generated by web analytics tools into business insight AND have broad knowledge across the entire scope of web analytics.

This is Why the WAA’s Educational efforts lack “tool focus”?

Sure. And of course, the tool vendors already own that focus, and by definition they have the resources to be much better at tool education / certification than the WAA, so why would the WAA want to compete with the tool vendors in the same space?

Better to add value on the business side, where there is demand we can fill and a lack of trusted resources. And if you think about it, this approach simply expands the overall WA opportunity. People who want to become experts on the tool side have a path (through the vendors), and people who want to become experts on the analysis / business side also have a path through the WAA. And if you want to be a Universal Web Analytics Soldier, I guess you could do both!

Does that mean I can pass the Test with No Tool Knowledge?

Not at all. The threshold we set is you need to be able to communicate effectively with tool experts to pass the test. That means you will need to know the basics of how the web works, how the tools accomplish their mission, and know what all the web analytics terms mean. Example: To pass the Test, you don’t need to know how to write a tag, but you do need to know when a custom tag is required and how to communicate your need effectively.

So Marketing people can Pass the Certification Test? eCommerce Managers? Usability people? Media Buyers? Etc.?

Absolutely, if they are good at transforming the data generated by web analytics tools into business insight AND have broad knowledge across the entire scope of web analytics.

Who Created the Certification Test and How?

About 50 WAA members from all over the world volunteered to take on the task. We created questions, tested them across different audiences, gathered feedback, rewrote the questions based on the feedback, tested the questions again. You know, the continuous improvement thing?

If you want to participate in the ongoing process of creating the Certification Exam, there is more info here. Please note you have to be a member of the WAA to be on any WAA Committee.

Where did the Requirements to take the Test Come From?

From the 4 Test the Test sessions we held at various eMetrics events, where we asked people to volunteer to take the Test. We looked at the backgrounds of people with high scores versus people with low scores and established the benchmarks. People with higher than average scores had these characteristics:

Years of Web Analytics Experience: 5.4
Interprets reports / suggests actions to be taken: 100% of population
Training / Courses in web analytics: 100% of population
Education post High School: 4.8 Years

People with lower than average scores had these characteristics:

Years of Web Analytics Experience: 2.3
Interprets reports / suggests actions to be taken: 50% of population
Training / Courses in web analytics: 63% of population
Education post High School: 3.6 Years

But inside these averages (segmentation!), it gets much more interesting. Turns out the less experience you have, the more formal education / training helps you get a higher score. Education could be college / advanced degrees, vendor training, or classes in web analytics / e-commerce. Logical, and expected.

Not so intuitive was this on the mix of education and experience: when you have a lot of one and little of the other, you tended to get a lower score. For example, both Ph.D’s with low years experience and people with 10 years experience but lacking education / training tended to get lower scores. Likewise, people who indicated they “read blogs and books” as the only source of education did not tend to have high scores unless they had a lot of direct web analytics experience. So somewhere in the middle there is a “magic mix” of experience and education that results in higher scores.

Interestingly, the single most reliable predictor of a higher score on the test was whether or not in the current job the person regularly suggests actions to be taken based on the analysis. This data point is more subjective than years of education or experience so we did not include it as a requirement to take the Test, but it’s worth mentioning since it aligns closely with the purpose of the test.

In the end, it’s tough to predict tangible business analysis skills based on just education or experience alone, and this is why the Certification Test should be an important tool for people hiring web analysts.

I’ve heard the Test is Difficult to Pass; can you Explain Why?

In short, because we are a young industry and people tend to have narrow experience relative to the scope of the topic.

You can be an expert in e-mail and Display analytics and still not pass the test because you don’t know enough yet about PPC analysis or Optimizing Web Sites. You don’t have to be an expert at everything to pass the Test, but you do need to have some knowledge across the entire scope of web analytics to get a high score. See the Knowledge Required for Certification document for an overview of topics.

That said, I’m sure many of you have been faced before with challenges you did not understand or have any experience with – and then you figured out how to produce insight. That brainset is precisely what the WAA is testing for. So if you can take what you know from e-mail analysis and use it to figure out a question about PPC analysis, you could answer the PPC question correctly. Do that enough times across the different knowledge areas and you could pass the Test, because you essentially demonstrated the ability to think analytically – the objective of the Test.

In opposition to that scenario, blindly following best practices in any knowledge area without recognition of the changes in approach a particular business situation or model might require means you probably will not pass the Test; you will need the capacity to modify your thinking based on the business goals presented. Example: the correct answer for the publishing model may not be the correct answer for the commerce model.

How Do I Decide if I Should Take the Test?

Honestly, I personally think the Certification has much more value to people who are in the earlier stages of their web analytics career. Let’s say you have the same training and read the same books as a lot of other folks. And you are trying to establish yourself as a person who can create business value but don’t have the resume to back that position up quite yet. Passing the Certification Test could give you the edge you need to make things happen faster for you.

Conversely, if you have an awesome resume of accomplishments and references for those deeds, then why would you need the additional “proof” the Certification Test provides? Plus, experienced people often specialize to distinguish themselves from the crowd, and a Test across the universe of Web Analytics would not be particularly relevant.

So I’d expect the majority of people taking the Certification Test to be say 3 – 4 years into their WA careers, or perhaps earlier if they have been focused on WA and exposed to the right training or experience environments when doing the actual work.

The above is from the perspective of an individual. However, an agency, consultancy, or service provider might decide having their analysts Certified (including senior people) creates a competitive advantage in their particular space. Companies looking outside for analytics help may feel more comfortable hiring a resource with WAA Certified talent on staff.

Are there any Questions?

Feel free to ask about anything, I probably am the person responsible for whatever you love or hate about the WAA Certification Test, and please see the WAA FAQ for questions on execution details.