How to collect high quality data

You can have the perfect research design, but if the data you collect isn’t good quality, you risk vague, inconclusive or even misleading results. You therefore risk wasting the time and resources you are investing to conduct the research. Here are some tips towards what you can do to make data collection a success:

Thorough planning

How well you plan your data collection is the biggest influencing factor on the quality of your data. Data collection is never simple, and more often than not issues pop up that threaten the quality of the data. Ensure you are prepared for this by planning every detail, for example through drawing up a detailed evaluation plan (create your own or use Project Oracle’s evaluation plan template to get started) and including information on:

• What information / outcomes will be measured with what tools? Draw up a table or matrix to keep an overview of what data serves what purpose. Make sure these are aligned with your research questions (the 1-3 key things you want to learn with your research project).
• Who is in charge of coordinating data collection? Ensure there’s someone to drive and take responsibility for the data collection.
• When exactly will data be collected? This may differ by outcome / tool. Set clear and realistic dates and define follow-up periods where relevant.
• Who will collect the data? Again, this may differ by outcome / tool. Have multiple conversations with potential data collectors to understand who would be suitable and available. Make sure the data collectors’ roles do not involve a potential conflict of interest that could affect or skew the data.
• Who will you be collecting data from? Clearly define your target population (e.g. age, geographical area, ethnicity etc.) to ensure you collect data from the right people.
• What will you do to keep response rates high? Come up with 2-3 strategies that feel suitable for your circumstances. This could involve sending reminders to participants, providing incentives (e.g. free lunch), removing barriers (e.g. providing assurances to counteract participants’ potential anxieties), or minimise required effort (e.g. a quick link to an online survey instead of a paper survey).
• How and where will the gathered data be stored? Ensure it is a safe place with restricted access to guarantee that data confidentiality cannot be breached. Ensure it is labelled accurately to avoid confusion, particularly later down the line. Ideally arrange it so it is easy to navigate and compile for data analysis.
• What kinds of analyses will be possible and appropriate for the data? This is important to inform your research design (e.g. if you need a comparison group or not). Try to be as specific as possible. Ask a data analyst or statistician if you’re not sure.

When you have your plan, ask someone with research experience to sense-check it. They may just find a gap in your plan or have a good idea or two on how you can improve it.

Use appropriate, accurate and reliable measurement tools

Whether they are surveys (click here to read about what makes a good survey tool), databases, interviews, focus groups or other types of data collection tools, they should be scrutinised for their suitability to reliably capture the insights you’re after. The easiest way to ensure your tool is reliable is to use a validated tool – a tool that has already been scientifically tested for its accuracy and reliability. They usually provide a variable called Cronbach’s alpha which tells you how reliable that tool is. As a rough guideline, a tool needs to have a Cronbach’s alpha of at least 0.70 to be deemed acceptable. However, if no validated tool is available for your project, it is advisable that you test your own tools before use – e.g. by collecting feedback from a test group of your service users on how appropriate it is, and by checking how meaningful the test data is towards answering your project’s research questions.

Ensure staff buy-in and preparedness

Take time to ensure your staff understand and ideally believe in the value of collecting data. Ensure they are clear about the exact process and their responsibilities in it, and that they are sufficiently resourced and trained to fulfil those responsibilities. Ensure they know how important it is to flag data collection issues, and who to speak to about any such issues.

Avoid changing your data collection approach half-way through the research project

No data collection approach is perfect, but changing it after data collection has begun usually does more damage to the quality of the data than sticking with a somewhat flawed approach. In fact, some flaws can be ironed out at the analysis stage.

Automate your data entry

By using online surveys or digital data entry templates, you can save substantial amounts of time on data entry. Similarly, you can drastically reduce the chance of inconsistencies occurring in the data (e.g. typos) and avoid needing more time to clean the data before it can be analysed.

Conduct data quality checks

Periodically review a sample of the incoming data to ensure you notice data problems as early as possible. This will allow you to act before these problems substantially damage the quality of your final dataset.

 

If you have any questions on any of the above tips, would like some help with your data collection plan, or would like to share your own tips from your experience with data collection, get in touch!

Get your downloadable version of this blog resource here!