The practical step-by-step guide
Open-ended survey questions are used to collect free format,
open responses to questions. They are
not restricted to a specific list of answers, so can provide a richness and
level of depth that closed-ended questions are unable to provide, as well as providing
insight into people’s emotions and in-depth descriptions of experiences in the
To learn more you can read our Complete guide to open-ended questions and consider the Top10 benefits of open-ended questions. Here
we explain the key stages of analysing open-ended questions and provide
guidance on the practicalities and benefits of using Excel or a dedicated
coding platform such as Codeit to process your coding project.
Before you dive in to start the process of analysing your
open-ended question, you’ll need to be familiar with The Seven Stages of
Analysing Open-ended Questions, which are summarised below:
Once you are familiar with these steps, covered in detail in
our separate article, you’ll need to consider which software tools to use. To help you decide whether
to use Excel - which you probably already have access to - or to dive in and
test a dedicated coding platform such as Codeit, we describe the processes you
would follow with each below.
We’ve provided examples throughout, using our survey
question, “Name something on your bucket list...” that we fielded with Google Surveys previously.
Usually, a survey tool such as SurveyMonkey will provide you
with a csv or Excel file containing your answers listed down the page, linked
to the survey ID.Considering the seven steps of analysing open-ended
questions described above, Excel can help you with many of these.
Within Excel, you can use the sort function and the filter
function to help understand the themes within the data in order to design
your codeframe. If your answers are
quite short, by sorting your data you will see some similar themes as you
scroll down the sorted list. Once you
have listed some of those themes, you can filter the data to show all mentions
of a code. Similar to sorting, the
filter function allows you to find multiple mentions of the same item, but can
also find further mentions of the item within a string of text. If you want to
go more advanced, you can try a pivot table, which will summarise
duplicate mentions within your codeframe.
If you’ve never used these functions, a quick Google search
will give you the step-by-step instructions on how to use them.
Once you’ve created a codeframe, to apply codes, you can
simply add a column next to your verbatim answers and begin to assign the codes
to the answers. Ideally use a code
number as reference for
ease. Starting with a simple codeframe
e.g. Australia (1), USA (2), you can begin to apply those codes in Excel.
Using the filter tool within Excel, you can easily find all
instances of ‘Australia’ and apply the codes.
Keep doing this until you have coded all you can, then you
can filter on the ‘code’ column, to bring up just ‘blanks’ (uncoded mentions)
to keep going until all IDs have codes applied.
If you have to apply more than one code to an answer, this
is where it gets more difficult in Excel. You can use commas to separate the answers, but analysing this data at the end can be difficult, unless you are using a
dedicated analysis platform to prepare your survey statistics and percentages.
Select a percentage of the answers and simply check if you
agree with the codes assigned. A simple
overwrite will update your coded answer column. If you need to log the fact that a quality check has been undertaken you could add another
column with a marker to show which have been checked.
Coding is absolutely possible within Excel, and has been the
tool of choice for those undertaking coding on a small scale for many
years. However, it is time consuming,
difficult to manage with multi-coded responses, and also difficult to then present
the resulting conclusions effectively. Here
are another eight
reasons why you might want to think twice before using Excel for coding.
Codeit can support all of the seven stages of coding - even the thinking part.
With one click, Codeit is able to display a summary of the
verbatim themes using its AI functionality. This cuts out a massive amount of thinking time, effectively creating
the framework for your codeframe automatically. You may want to spend a bit of time refining the autogenerated result, but it significantly reduces the time taken all the same.
When it comes to coding, one size definitely does not fit
all. Codeit provides three different,
interchangeable tools which you can use in a blended way to code your data accurately and quickly. Codeit’s "Short Text" mode for example, (shown below) automatically groups similar text into codeframe suggestions for the user to review.
The traditional approach to coding requires that you create most of
your codeframe upfront in advance. Codeit however, gives you the best of both worlds - you can define codes upfront and/or build up your codeframe on-the-fly as you go. A simple right-click enables you
to add a code to your codeframe. You can also add codes directly within a net in your codeframe and make refinements to the net structure (move codes between nets, add new nets/subnets etc..). Often, much of the skill in coding is in fluidly evolving and refining the codeframe as you go - so having a tool to support this is very useful.
Codeit allows you to code in a much more structured way than
Excel without the worry of error. Further, if you continue your survey over several months, Codeit can
learn how you code in order to use this learning and automatically code in
future. For example, if it sees you code
“Go to Sydney” as “Australia” a number of times, when it sees the verbatim in
future it will be auto-coded with no manual effort whatsoever.
Codeit has all the features to enable you to perform the
required quality checks and record them within the system for auditing. The dedicated ‘Verify’ mode has features to
quickly and easily select a percentage for checking, and apply approvals.
Once you’ve understood the key steps required for good
quality analysis of open-ended survey responses, you can choose to use Excel or
try a free trial of a dedicated coding platform like Codeit. Excel is familiar, and does have useful tools
that can help you code, but there will always be limitations and awkward challenges when using this tool. Taking the step to use a dedicated
platform for analysing your open-ended survey questions will reap rewards, in
time saved, improved accuracy and in maintaining quality.
you’d like to give Codeit a go, you can take up their 30
day free trial
to help you start coding and analysing your open-ended question data today.
Try it for Free
Anything we can help you with? Ask us