What Data Do You Already Have, and Is It Good Enough?
Most mobile health programs collect a great deal of data. It can still be difficult to say exactly what the program holds, where the data live, how complete they are, or whether staff define the same field in the same way.
The number of fields matters less than whether the data are defined and collected consistently. Three reliable measures may answer a research question better than forty fields used differently across sites.
Start by taking stock. Then judge quality against the question you want to answer, not against a general standard.
Find everything first
Data rarely sit in one place. Before judging their quality, list every source, including informal ones.
- The electronic health record, if you use one, including your own and any parent organization’s system
- Scheduling and route systems
- Paper intake, consent, and screening forms
- Spreadsheets kept by staff for their own tracking
- Grant and funder reports already produced
- Registration or eligibility systems
- Partner and site host records
- Referral logs and follow-up trackers
- Patient experience surveys
- Vehicle, staffing, and operational logs
For each source, record what it covers, who enters it, how far back it goes, where it is stored, and who can get to it.
A tracking spreadsheet kept by one staff member is still a data source. It may be the only place a particular measure exists, and the program may lose access to it when that person leaves. Include these files in the inventory and move them to an approved location the program controls.
Judge quality against five questions
Quality is not one number. Ask five things about each measure that matters to you.
Is it complete? What share of eligible visits or patients have a usable value? A field completed for only a fifth of visits may not support the planned analysis, however important the measure is.
Is it consistent? Would two staff members at two sites record the same situation the same way? Free-text fields, optional questions, and locally invented categories are where consistency usually breaks.
Is it accurate? Does the recorded value match what happened? A field may be less reliable when the system requires it but staff do not use it in care or understand its purpose.
Is it timely? How long between the event and the record? Data entered days later, from memory or from paper, carry different errors than data entered at the point of care.
Is it usable? Can you obtain it in a form you can analyze? A measure may be well recorded but still unavailable for evaluation if the program cannot export or report it. See Getting Your Data Out of a Health System EHR.
Do a quick completeness check
You do not need a formal audit to learn most of what you need.
- Choose the five to ten measures closest to the question you care about.
- Take a defined period, for example the last three months.
- For each measure, count how many eligible records have a usable value.
- Note the share, and note where the gaps cluster by site, staff member, service, or day.
The pattern of missing data can point to the cause. Gaps concentrated at one site, with one service, or during one workflow may indicate an operational problem. Gaps found throughout the data may indicate that the field is unclear, difficult to use, or not viewed as useful. Ask the staff who collect the data before deciding what to change.
What to do when quality is poor
Poor data quality is common. It does not necessarily end an evaluation, but it may change the question, design, or collection plan.
Narrow the question
Choose a question the available data can answer. Reach and access measures may be more complete than clinical outcomes because programs often record patients and visits more consistently than clinical detail. Confirm this pattern in your own data rather than assuming it.
Fix forward rather than backward
Cleaning historical records is slow, and often impossible where the underlying information was never captured. Improving collection from a fixed start date is usually faster and more reliable. Say plainly which period is analyzable and why.
Define before you collect
Many quality problems begin with an unclear definition. Write the exact numerator, denominator, eligible population, and permitted values before changing the form or system. See Data Dictionaries for Mobile Health Programs.
Reduce what you collect
Review whether every field has a clear purpose. Removing fields that no one uses can reduce burden and make it easier for staff to complete the measures that remain.
Improve the workflow
A field may be skipped because it is hard to find, unclear, asked at the wrong moment, or has no visible purpose. Ask the people entering it what gets in the way before adding training or reminders.
Collect a small amount prospectively
If the measure you need does not exist, a short prospective collection on a defined sample can answer the question faster than reconstructing history. Keep it small, define it precisely, and set an end date.
Write down what you find
Record the limitations you discover, in plain language, as you go. Which measures are reliable, from what date, for which sites, and with what known gaps.
This summary gives a researcher a realistic account of the data at the first conversation. It also reduces the need to repeat the assessment for every new partner and provides the starting point for a data dictionary.
Frequently asked questions
How complete does a measure need to be?
It depends on the question, analysis, amount of missing data, and reasons values are missing. The same completeness rate can have different implications when the gaps are concentrated in one site or population. Report the rate and pattern of missing data with the findings rather than relying on one universal threshold.
Should we clean our historical data before talking to a researcher?
Not necessarily. Begin by sharing what is known about the data and their limitations. A researcher may be able to design around a known problem or help identify which cleaning would materially improve the analysis.
Our data live in someone else’s system. Where do we start?
Start with the inventory anyway. Knowing what exists and who controls it is the first step in getting access, and it is the information the data team will ask for. See Getting Your Data Out of a Health System EHR.
Is a spreadsheet a legitimate data source?
Yes, and it is often the only source for a measure the program cares about. Treat it as a real system: define the fields, agree who maintains it, keep it somewhere the program controls, and address privacy the same way you would for any record containing patient information.
Related resources
Have a research question or a program worth studying?
GMHRC helps researchers and mobile healthcare programs find each other and plan work that is useful to both.