Collaborative coding of qualitative data

Coding qualitative data is a huge undertaking. Even with relatively small datasets, it can be a time-consuming and intensive process, and relying on just one person to interpret complex and rich data can leave out alternative viewpoints and risk key insights being missed. For one or both of these reasons, qualitative analysis is often performed as a collaborative team, with multiple coders either splitting up the task or providing multiple interpretations and checks on analysis. It is something that Quirkos users have been requesting for a long time, and we are so excited to be releasing it this month. However, let’s look into the reasons that researchers might want to share the coding and analysis process, and some practical and methodological considerations.

There’s a great chapter in the sage Handbook of Qualitative Data Analysis on collaboration by Cornish, Gillespie and Zittoun (2014). First, they ask who is involved in the analytic collaboration? It could be academic-academic, or involve practitioners, and anything from two to a dozen people. It can also involve ‘lay-persons’ through participatory analysis, something we’ve written about before. Collaborators may also have different roles, for example there may be primary and secondary coders in a hierarchy, or they may have the same role and status in the decision making.

Work can also be assigned in many different ways, depending on the volume of data, researcher time and methodological approach to validity. Sometimes more than one coder will analyse all of the dataset for validity, or sources may be split up so that they only have to code part of the data – especially when there are a large number of sources. Alternatively, another researcher could be used as ‘sense-checking’, reviewing already coded work to see if anything was missed, or questioning interpretation. It’s unusual to split work by theme, with one person coding a single topic area. Since qualitative data has to be read through completely anyway, coders may as well work with all the topics and themes in the project.

Enacting collaborative coding is a process to be managed all of it’s own. Richards and Hemphill (2018) describe this in 6 stages, essentially planning the practicalities of how researchers will contribute, designing a codebook or framework, testing the codebook before coding, then finally a review of the process and results. These basic steps are the same regardless of the analytical approach, and while Richards and Hemphill (2018) describe open and axial coding, pretty much any coding approach will work (eg IPA, grounded theory, thematic or discourse analysis). Often a codebook or coding framework is agreed before coding starts, with all researchers discussing which themes they want to look for in the data, how they will be worded and defined, based around answering the research questions.

However, when using grounded theory or other approaches where codes/themes are generated during the course of the analysis, there is extra need for reviewing codes or themes as they develop. This also needs a set of rules, deciding when new codes will be created, if everyone can do it, guidelines for when to merge or group codes to themes etc.

There is also deciding how collaboration will happen – will it be in person? Remotely? If so, over the phone? Video chat? Directly in qualitative analysis software? The practicalities of this (especially distance and capabilities of the software) will dictate much of what is realistic, but note that collaboration could involve anything from a long structured workshop session to a chat over coffee.

The new live collaboration feature in Quirkos Cloud offers a unique and flexible way to collaborate on qualitative projects. With data stored remotely on the cloud, users can login from any computer, access the project, view, edit and discuss in real time. This eliminates the need to ‘manage’ project files: knowing who has edited a file and when, where it will be stored, and who has worked on it.

However, it also allows flexibility for researchers to contribute to the project as and when they have time. It doesn’t require everyone to work on the project at set times, and this is very useful when working across time-zones (or teaching schedules!).

The live chat feature in Quirkos Cloud is a totally unique feature to Quirkos, that a lot of researchers are excited about. Part of the challenge in analysing qualitative data as part of a team is having a way to log changes and decisions, and to discuss issues with others. The chat function allows teams to do this in a familiar and intuitive way, recording and saving the conversation, user and time of each comment. This acts as an easy to read playback of the analytics process, as well as a place for more informal communication and discussing practicalities or the process.

But the chat feature is also available in the static offline projects too (since they have identical files and features). This means that you can also use the ‘chat’ for a reflexive coding journal, or to keep project level notes and analytic writing attached to the project. I’m really interested to see the different ways users use this new functionality!

There are a couple of other software tools that allow collaboration, but we think that Quirkos Cloud will be the easiest to work with. Nvivo Teams requires server installation, and although pricing is not publicly available, it’s eye-wateringly expensive. Dedoose is a much better cloud platform for collaboration, but it’s reliance on installing unsecure versions of Flash and Air mean that it’s becoming difficult to run.

The collaboration features in Quirkos are free for cloud users, and there is no limit to the number of projects you can share, and the number of users on each project. It’s easy to share a project (you only need the user’s email address), and you can give full editing or just read access to each person as needed. All the projects are securely saved on our cloud servers so the whole team can access them from anywhere, but you always have the option to download the project and work with it offline.

Using the query view, you can get side-by-side views of different team members coding, to see how you have differed and where everyone agrees. While we don’t provide any quantitative measures for inter-rater reliability or agreement, (more on this in a future blog post) it’s still easy to find and discuss differences.

Yet the main benefit will be how whole teams can now collaborate using the simple and visual Quirkos interface. This means it’s quick for everyone to learn, fast to use while coding, and just plain fun to see the highlights and bubbles growing in real-time as people work on the projects. It really connects you to your project and team much more than passing around a static document. The full feature set will be available in Quirkos 2.3 (out in late Feb 2020), but you can try the intuitive visual Quirkos approach to qualitative analysis with a free trial of the offline or cloud version from https://www.quirkos.com/get.html or read more about the features here.

References

Richards, K. R., & Hemphill, M. A. (2018). A Practical Guide to Collaborative Qualitative Data Analysis, Journal of Teaching in Physical Education, 37(2), 225-231. Retrieved Feb 7, 2020, from
https://journals.humankinetics.com/view/journals/jtpe/37/2/article-p225.xml

Cornish, F., Gillespie, A. & Zittoun, T. (2014). Collaborative analysis of qualitative data. In Flick, U. The SAGE handbook of qualitative data analysis (pp. 79-93). London: SAGE Publications Ltd
https://methods.sagepub.com/book/the-sage-handbook-of-qualitative-data-analysis/n6.xml