This is a post written by Paul Watson, an Atlassian technical writer, as part of an ongoing series written by the technical writing team, exploring the latest techniques in technical communication. We’ll write about our projects, experiments and ideas, and we’ll share the techniques we use to give our customers the sweetest documentation in the world.
Getting Customer Feedback
There’s no question that getting feedback is a good idea – a law of nature even. Think survival of the fittest. Think very large, very extinct, reptiles. In the information realm, think Britannica.
“In 1933, the Britannica became the first encyclopaedia to adopt “continuous revision”, in which the encyclopaedia is continually reprinted and every article updated on a schedule. In March 2012, Encyclopædia Britannica, Inc. announced it would no longer publish its printed editions, instead focusing on its online version.” From Wikipedia (and so rich with irony).
Hmm… continuous deployment 80 years ago!
You could also see feedback as part of the ‘fail fast’ philosophy – get it out there and see how it flies.
So, I’ve been wondering about how we get feedback on the Atlassian product docs that are publically available at confluence.atlassian.com (CAC). We’ve all seen the big guys do this sort of thing…
Tech writers, and others, have tried various ways of getting feedback. Quoting very selectively…
“I’ve engaged them directly (support tickets, email, phone, and f2f), done surveys, used feedback forms, and used web analytics tools. They can all yield different and interesting bits of information…” Richard S. on Technical Writing World forum
Actually, we already get feedback on the Atlassian product docs. When you look at the bottom of a CAC page, there are 4 channels for ‘feedback’:
- add a page comment
- log a Support request
- raise a Jira issue
- ask the community for Answers.
(You can also sign up to contribute to the docs directly.)
And on the Atlassian developers site (developer.atlassian.com, or DAC) you can use a feedback form:
When you click the tab thingy you get a form that allows focussed feedback on that documentation page. Nice! And the implementation is interesting – a plugin in Confluence provides a customisable form, and adds the feedback as an issue to a Jira project.
To get some perspective on these channels, I gathered numbers for February 2012:
|For Feb 2012||URL||Activity||Doc Feedback|
|Answers(since June 2011)
So, we got about 400 items of ‘feedback’ in February (not counting Answers activity). It’s probably not meaningful to compare rates, but if this constitutes a ‘funnel’ in the GA sense, it would be interesting to have numbers for clicks on those CAC page buttons.
However, it is clear that we get docs feedback all over the place. It’s also clear that none of these channels, except for the DAC form, provide systematic, focussed feedback on a doc page as documentation. That makes it hard for us to have a well-defined way to process this feedback. So the feedback we do get may not be being used as well as it could… I know that when I’m revising a page on CAC, I don’t check all of those locations for comments.
I want to call out two of the above approaches for getting feedback on product documentation:
- Page comments
- A focussed feedback channel.
In some situations, page comments work really well. For example, on Atlassian’s intranet, where there is a community of people who are engaged with the specific issues raised, we frequently see long fruitful discussions in the page comments.
On CAC, a public site, people mostly comment because they are having trouble with a product. Here are typical (I get to chose what that means) comments from one page in the Confluence docs:
“How can I create an easy link on a Confluence page that will allow them to Export as PDF a select set of pages? I don’t want to make new intranet users figure out the multi-step process…” Meg Sept 2010
This isn’t really a comment about the documentation – Meg is asking about functionality that doesn’t exist in Confluence, and so isn’t documented. She is using page comments as a kind of forum thread based on the page topic, although comments like this would probably get a better response on Answers. And she expects some response, probably from Atlassian (we’ve had some really harsh comments when that doesn’t happen). She isn’t reporting a bug, but there’s a feature request lurking in there too… Page comments become a kind of catch-all for all kinds of feedback.
Of course we do get page comments that point out issues with the page as documentation:
“It looks like 3.2 for linux also uses
conf/wrapper.conf. The docs should be updated to say so.” Anon
This type of comment, however, is relatively uncommon.
A dedicated channel for focussed feedback provides a number of interesting possibilities, because it gives the tech writers some control over how the feedback is solicited and processed. Because the reader knows that they are providing feedback about the page, they are unlikely to use this channel if they actually want to request a feature. Furthermore, feedback collected in this way is potentially easier to process in various ways. That gives the tech writers the opportunity to:
- mine the feedback on a page, when I’m working on the page.
- find out which are the worst ranked pages, and why people dislike them.
- find out which are the best-ranked pages, and why.
- match up feedback with GA data, to help inform the stats.
Feedback could possibly be targeted in various ways. You might only collect it for particular doc changes, or turn it on just at certain times (for example, after releases). You could try A/B tests (for example, using different documentation styles for different products). Feedback doesn’t have to be on for all product docs all the time.
What about you? Do you ever want a direct way to comment on the docs, as docs, or are you happy just to tag a page with a comment? You can give feedback by dropping a comment on this post!