|Year : 2022 | Volume
| Issue : 4 | Page : 443-447
Electronic versus paper-based data collection for conducting health-care research: A cost-comparison analysis
Sirshendu Chaudhuri1, Bhavani Shankara Bagepally2, Ditipriya Bhar1, Uday Kumar Reddy Singam3
1 Consultant, Division of Online Courses, ICMR National Institute of Epidemiology, Chennai, Tamil Nadu, India
2 Scientist E, Division of Non-Communicable Diseases, ICMR-National Institute of Epidemiology, Chennai, Tamil Nadu, India
3 Resident, Department of General Surgery, Apollo Institute of Medical Sciences and Research, Chittoor, Andhra Pradesh, India
|Date of Submission||31-May-2021|
|Date of Decision||08-Sep-2022|
|Date of Acceptance||16-Oct-2022|
|Date of Web Publication||31-Dec-2022|
ICMR-National Institute of Epidemiology, Chennai, Tamil Nadu
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: Containing expenditure and efficient resource use is essential to limit the increasing costs of health research. Electronic data collection (EDC) is thought to reduce the costs compared to paper-based data collection (PDC). Objectives: As economic evidence in this area is scanty, especially in low- and middle-income countries, the objectives of the study are to perform an economic evaluation and compare the cost between EDC and PDC. Methods: A cost-comparison study was conducted to compare between EDC and PDC from the institutional perspective for the year 2018, based on a community-based survey. Step-down cost accounting was adopted with a bottom-up approach for cost estimation. Total and unit costs were estimated with the base case comparison between EDC and PDC while using SPSS software (e-SPSS and p-SPSS, respectively). We conducted scenario analyses based on the usage of different software, R and STATA for both EDC and PDC (e-R, p-R, e-STATA, and p-STATA, respectively). One-way and probabilistic sensitivity analysis (PSA) was performed to examine the robustness of the observed results. Results: In the base-case analysis, total costs of EDC and PDC were ₹72,617 ($1060.9) and 87,717 ($1281.5), respectively, with estimated cost reduction of ₹15,100 ($220.6). In other scenarios, the estimated cost reduction for e-R, e-STATA, p-R, p-STATA was ₹−274 ($4.0), 98 ($1.4), 14826 ($216.6), and 15,002 ($219.2), respectively, when compared to EDC-SPSS. On one-way and PSA, the results of the cost-comparison analysis were robust. Conclusion: EDC minimizes institutional cost for conducting health research. This finding will help researchers in efficiently planning for the budget for their research.
Keywords: Cost comparison, cost-estimation, cost-minimization analysis, data collection, paper-based data collection
|How to cite this article:|
Chaudhuri S, Bagepally BS, Bhar D, Reddy Singam UK. Electronic versus paper-based data collection for conducting health-care research: A cost-comparison analysis. Indian J Public Health 2022;66:443-7
|How to cite this URL:|
Chaudhuri S, Bagepally BS, Bhar D, Reddy Singam UK. Electronic versus paper-based data collection for conducting health-care research: A cost-comparison analysis. Indian J Public Health [serial online] 2022 [cited 2023 Feb 1];66:443-7. Available from: https://www.ijph.in/text.asp?2022/66/4/443/366562
| Introduction|| |
A sustainable health research environment requires an integrated contribution from individual, institutional, national, and international solidarity. A major proportion of health research costs in low- and middle-income countries (LMICs) is contributed by the donor funding and grants. However, available funding resources are much limited in comparison of what is required. Therefore, the cost of research can be well managed by systematic planning and implementation as well as efficient use of resources during data collection, management, and analysis.
Paper-based data collection (PDC) using paper case report forms (CRFs) are the conventional method of data collection in health research. In addition, PDC can be a laborious and error-prone process. With the innovation of information technology, an increasing trend of use of electronic data collection (EDC) has been observed in many regions of the World. Evidence suggests that in LMIC settings, EDC may serve as an effective platform to collect and transfer large quantities of data successfully. In India, although systematic evidence on the use of EDC is lacking, the Indian government's policy on the use of mhealth for service delivery by health workers is an indication of an increasing trend in the application of EDC. With the advent of Health Information Technology in India, there is an increasing trend of EDC by small to extensive scale surveys and trials than traditional PDC in researches.,,
The current literature from different country settings has identified various advantages of EDC when compared to PDC. It is reported that EDC ensures increased accuracy, greater efficiency of data capture, better quality control of data management, rapid analysis of data, as well as a reduction in the cost of data collection.,,,, However, the cost of EDC and PDC can be contextually different and study specific. Despite the rapid increase of EDC in India, there is hardly any study examining the cost difference between the two methods. Hence, based on a cross-sectional survey, an economic evaluation from the institute's perspective was conducted to examine the costs incurred in different modes of data collection and methods applied while conducting health research and to examine the cost difference between the costs of traditional PDC and EDC.
| Materials and Methods|| |
This study was conducted by a cost-comparison analysis approach. This economic evaluation is based on a single community-based survey in the state of Andhra Pradesh, India. This survey was conducted in a rural area of Andhra Pradesh from a private medical college. The survey was conducted from May to September 2018. Data for this study were collected electronically through an android device using KoBoCollect open data kit.
Retrospectively, costing was assigned to this survey. For costing, a step-down cost accounting was adopted with a bottom-up approach. All the costs were estimated from an institutional perspective. We identified different cost centers and cost components within each of the cost centers for various activities/processes involved in the study. Then we imparted costs for each of the components. All the costs were estimated in Indian Rupee (₹) for the year 2018 (Also converted into the United States Dollar; conversion rate: 1$=68.45₹ as on July 1, 2018). As the study was of a short period (<1 year), we have not discounted costs.
Cost comparison is the primary outcome of this study. Data collection for the original survey was done electronically, and the analysis was done using the Statistical Package for Social Sciences Software (SPSS) Version 25.0. Armonk, NY: IBM Corp' after '(SPSS)' so that the whole sentence reads as-”Data collection for the original survey was done electronically, and the analysis was done using the Statistical Package for Social Sciences Software (SPSS), Version 25.0. Armonk, NY: IBM Corp. Similarly, we have imparted costs for all the relevant cost components for PDC, with a fixed study sample size, the same as that of EDC. Thus, we estimated costs of the survey (p-SPSS) assuming that PDC would have been done instead of EDC in relevant cost centers. Then, the cost difference was estimated as the difference in total costs between EDC and PDC mode of data collection.
Then for different scenarios of conducting the same study using different data analysis software was considered. For scenarios, we estimated the costs of the survey if other software such as “R” and “STATA” would have been used for data analysis for both EDC and PDC approaches. These costs were termed e-R and e-STATA when EDC was done, and data analysis software was R and STATA, respectively. Similarly, costs were termed as p-R and p-STATA when PDC was done with analysis using R and STATA, respectively. To examine the uncertainty of results, a one-way sensitivity analysis was conducted by lowering or increasing the individual cost components by 25%. Furthermore, probabilistic sensitivity analysis (PSA) was conducted by simulating the various components of the cost data using a gamma distribution. Based on the assumption that, if the same survey was conducted in low-, medium-, and high-cost settings, all the respective cost components were documented [Supplementary Table 1]. For example, the cost of a medical college can be situated in a rural setting (low-cost setting), small towns and cities (medium-cost setting), or in metropolitan cities (high-cost setting) where the costs of the same items can vary. Using these estimates from three different settings, the measures of dispersion were calculated and used as input while conducting the PSA. All the analyses were performed in Microsoft Excel 2019.
The institutional ethics committee clearance has been obtained to conduct the study (Ref No: IEC04/AIMSR/02/2018).
| Results|| |
The primary study was conducted with a sample size of 102 subjects. The steps of the survey were broadly divided into three phases – preparation, data collection, and data analysis and storage [Figure 1].
All identified cost centers and its component costs for both EDC and PDC modes of data collection are tabulated in [Table 1]. The costs of the preparatory phase and the data collection phase were similar for both EDC and PDC, with minimal variation between the two. A significant difference in costs between EDC and PDC was noticed in the costs related to data entry and CRF storage section [Table 1].
|Table 1: Cost centers and its component costs for electronic and paper-based data collection|
Click here to view
The estimated total costs of EDC-SPSS and PDC-SPSS were ₹72,617 and ₹87,717, respectively, with estimated cost difference of ₹−15,100 [Table 2]. The data analysis and storage section of cost centers contributed ₹14,154 (93.7%) of total cost difference. In scenario's analysis, estimated cost difference for EDC-R, EDC-STATA, PDC-R, and PDC-STATA were ₹−274, ₹−98, ₹14,826, and ₹15,002 respectively when compared to EDC-SPSS. In the one-way sensitivity analysis, the cost difference between EDC-SPSS and PDC-SPSS was robust with maximum variation <8% even with a 25% change in the individual component costs. The most sensitive cost inputs were transportation-related vehicular costs for data collection, the salary of the researcher, and the costs of data analysis [Figure 2]. The one-way sensitivity analysis results of all the scenarios were robust with minimal deviation from that of observed base case results [Supplementary Figure 1], [Supplementary Figure 2], [Supplementary Figure 3].
|Figure 2: One-way sensitivity analysis for comparison between e-SPSS and p-SPSS.|
Click here to view
In PSA [Supplementary Figure 4], results were robust as the PSA mean cost difference between e-SPSS as compared to p-SPSS was similar to that of base case values [Table 2]. Similarly, even in the PSA results in different scenarios were also robust and similar to that cost difference from the respective-based case analysis. Since the costs of software (R and STATA) could not be varied; hence, we could not calculate the measures of dispersion and hence both PSA and base-case values were the same. Results of PSA of cost-comparison analysis were shown in a scatter plot with the X-axis indicating the total costs of the comparator (p-SPSS) while the Y-axis was indicating the comparators (e-SPSS, e-R, e-STATA, p-R, p-STATA) [Supplementary Figure 4].
| Discussion|| |
In this economic evaluation, we have examined the cost comparison that could result from the use of EDC using open-source electronic questionnaires when compared with traditional PDC. We observed that the use of EDC reduces the total cost of health research by 17%. Higher costs of PDC were mostly (93.7%) contributed by the cost of data entry and CRF storage. This finding holds good for commonly used software for data analysis.
Estimating the cost-comparison due to EDC is complex and study specific. One study from South Africa estimated that EDC could reduce the study cost by 50% over the traditional PDC. Contemporary evidence from other parts of the world also indicates similar findings that the cost-comparison can vary between 49% and 62%. This difference could be at various levels of the study. In the present study, we found more than 90% of the estimated cost-comparison was out of double data entry and the storage of paper case record forms (CRFs). The process involved with tedious data entry, and the necessary data cleaning, reduces the duration of health research and resulting costs involved in it.,, Evidence from LMICs such as Ethiopia and Tanzania has shown that the use of EDC reduced the costs by 25%., However, the economic evaluation from the Ethiopian study considered the cost of electronic devices as single use, which may cause an over-estimation of cost for EDC.
Although the cost of the data collection device did not influence the cost difference, the choice of device may affect the cost difference. The types of electronic devices may affect the accuracy of the data collection as well in the field due to variation in size, and specification. The estimated cost difference can be affected by the initial expenditure associated with the establishment of a system in an institution for using EDC methods instead of PDC., For example, we used an open-source tool for developing electronic questionnaires in our study. The use of a paid version of a similar tool could have altered the estimate.
The present study highlights various costs incurred to conduct the study from the institutional point of view at various stages of short-duration, community-based survey. Such findings will help in planning for the necessary set-up required by an institution to adopt the EDC approach instead of PDC in health research. The initial planning includes motivating and training the researchers, technologies needed, financing, maintaining the technology, and solving the ethical concerns associated with handling the data.,, Once an electronic data capture system is established in an institution, it can save additional resources like costs, and the researchers can focus better on their research activities.
In the present study, we could not account for the costs in a few aspects of health research. These include the cost associated with data cleaning in case of PDC, maintenance charge during storage of data collection. We could not impart such charges as the original study sample size is relatively small and thus finding the present study should be examined cautiously for larger studies. As we use an open-source tool for data collection, the actual cost may vary if an institution could use a paid tool for EDC. The study explored the costs of public health research. Further study observations may not be extended for clinical trials.
| Conclusion|| |
We observed a cost reduction with the use EDC approach as compared to the PDC approach while conducting a health research cross-sectional single-center study. For short studies, we strongly recommend using EDC. We also recommend that the research institutions should priorly assess the potential cost of preparation and maintenance related to technologies associated with EDC by incorporating all the stakeholders. Further cost-minimization analysis is warranted for clinical trials and multi-centric studies in this area to have a better policy decision by all stakeholders.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
WHO Commission on Macroeconomics and Health & World Health Organization. Macroeconomics and Health: Investing in Health for Economic Development: Executive Summary/Report of the Commission on Macroeconomics and Health. World Health Organization; ↱2001. Available from: https://apps.who.int/iris/handle/10665/42463
. [Last accessed on 2020 Mar 26].
Thriemer K, Ley B, Ame SM, Puri MK, Hashim R, Chang NY, et al.
Replacing paper data collection forms with electronic data entry in the field: Findings from a study of community-acquired bloodstream infections in Pemba, Zanzibar. BMC Res Notes 2012;5:113.
Dillon DG, Pirie F, Rice S, Pomilla C, Sandhu MS, Motala AA, et al.
Open-source electronic data capture system offered increased accuracy and cost-effectiveness compared with paper methods in Africa. J Clin Epidemiol 2014;67:1358-63.
Patil R, Roy S, Ingole V, Bhattacharjee T, Chaudhary B, Lele P, et al.
Profile: Vadu health and demographic surveillance system Pune, India. J Glob Health 2019;9:010202.
Bhattacharya S, Kumar A, Kaushal V, Singh A. Applications of m-Health and e-Health in public health sector: The challenges and opportunities. Int J Med Public Health 2018;8:56-7.
Singh T, Roy P, Jamir L, Gupta S, Kaur N, Jain DK, et al.
Assessment of universal healthcare coverage in a district of North India: A rapid cross-sectional survey using tablet computers. PLoS One 2016;11:e0157831.
Gupta SK. Paperless clinical trials: Myth or reality? Indian J Pharmacol 2015;47:349-53.
] [Full text]
Dickinson FM, McCauley M, Madaj B, van den Broek N. Using electronic tablets for data collection for healthcare service and maternal health assessments in low resource settings: Lessons learnt. BMC Health Serv Res 2019;19:336.
Rorie DA, Flynn RW, Grieve K, Doney A, Mackenzie I, MacDonald TM, et al.
Electronic case report forms and electronic data capture within clinical trials and pharmacoepidemiology. Br J Clin Pharmacol 2017;83:1880-95.
Carvalho JC, Bottenberg P, Declerck D, van Nieuwenhuysen JP, Vanobbergen J, Nyssen M. Validity of an information and communication technology system for data capture in epidemiological studies. Caries Res 2011;45:287-93.
Avilés W, Ortega O, Kuan G, Coloma J, Harris E. Integration of information technologies in clinical studies in Nicaragua. PLoS Med 2007;4:1578-83.
Reddy SU, Chaudhuri S, Amrin SF, Rabeka G. Nutritional status and dietary intake among unmarried adolescent girls in rural area of Chittoor district: A community based survey. Int J Community Med Public Health 2019;6:4034-42.
KoBoToolbox. Data Collection Tools for Challenging Environments. Available from: https://kobotoolbox.org/
. [Last accessed on 2020 Mar 26].
Pavlović I, Kern T, Miklavcic D. Comparison of paper-based and electronic data collection process in clinical trials: Costs simulation study. Contemp Clin Trials 2009;30:300-16.
Walther B, Hossin S, Townend J, Abernethy N, Parker D, Jeffries D. Comparison of electronic data capture (EDC) with the standard data capture method for clinical trial data. PLoS One 2011;6:e25348.
Welker JA. Implementation of electronic data capture systems: Barriers and solutions. Contemp Clin Trials 2007;28:329-36.
Brandt CA, Argraves S, Money R, Ananth G, Trocky NM, Nadkarni PM. Informatics tools to improve clinical research study implementation. Contemp Clin Trials 2006;27:112-22.
King JD, Buolamwini J, Cromwell EA, Panfel A, Teferi T, Zerihun M, et al.
A novel electronic data collection system for large-scale surveys of neglected tropical diseases. PLoS One 2013;8:e74570.
Ventola CL. Mobile devices and apps for health care professionals: Uses and benefits. P T 2014;39:356-64.
Haller G, Haller DM, Courvoisier DS, Lovis C. Handheld vs. laptop computers for electronic data collection in clinical research: A crossover randomized trial. J Am Med Inform Assoc 2009;16:651-9.
Pace WD, Staton EW. Electronic data collection options for practice-based research networks. Ann Fam Med 2005;3 Suppl 1:S21-9.
Ayatollahi H, Mirani N, Haghani H. Electronic health records: What are the most important barriers? Perspect Health Inf Manag 2014;11:1c.
Mosa AS, Yoo I, Parker JC. Online electronic data capture and research data repository system for clinical and translational research. Mo Med 2015;112:46-52.
[Figure 1], [Figure 2]
[Table 1], [Table 2]