A Frame-Based Case Report Form Tracking System S. Copping, E. Otun, D. Durrieu Rhone-Poulenc Rorer France
INTRODUCTION This presentation describes the system we use to manage the data entry function within the Clinical Data Management Department. Planning data entry is a problem in Data Management. Work in the form of patient dossiers (Case Report Forms or CRFs) is received at an uneven rate. Some days hundreds of CRFs come in while on other days there are none at all. Additionally, the amount of resources needed changes according to the time of year and to the number and stage of the pharmaceutical products under development. Company policy is to employ sufficient data entry operators to cover the basic workload. During peak periods, we employ temporary staff or send work out to contractors according to a priority system. This implies a means of predicting when the in-house team will be saturated so that appropriate measures may be taken. Planning was done on an empirical basis until recently when we realized that the humble logfile we had been using for years to track CRFs within the department could serve as the foundation of a more sophisticated system. The logfile recordsCRFs received and entered. It consists of two separate but identical files, an active file for ongoing studies and a history file for finished studies. It contains the study and patient numbers, the date of reception, the dates of first and second entry and identifies the data entry operators. It is now about 10 years old and so contains a fair amount of information useful for calculating rates of reception and entry. It is maintained daily by a technician whose primary task is to keep it up-to-date. We were discussing how to. improve data entry planning when we received SAS® version 6.08. We decided to use the new features in SAS/AF® to develop a more sophisticated tracking system that would be similar to an Executive Information System. Indeed, many of the ideas for this tool came from the SASIEIS® module.
THE SYSTEM The system requires a PC (preferably a 486) with a graphics printer connected to a Vax® via Decnet®. Data are stored on the central Vax to save disk space on PC and to enable access by several users. The application is written in SAS/AF® and currently consists of two major components. A data entry facility and an executive user facility. The features discussed in this presentation are those for the executive user, a subset of these will shortly be made available to Data Managers within the department to help them follow their own projects.
905
FIG 1 - THE EXECUTIVE MENU
DATA MANAGEMENT TRACKING SYSTEM
Fig. 1 shows the Executive Menu, that is the menu the executive user sees on entering the system. This contains five options. 'Key Figures' allows access to update data entry rates for a specified study or project based on the most recent data available while the 'Rates' option gives details of the performance of individual operators. 'Data Preparation' and 'External DE' are new to this, the second version of the system (DE is an abbreviation for Data Entry). The heart of the application and historically the first part developed, is the backlog menu. THE BACKLOG MENU
This screen allows the user to select the project that interests him. It then displays two critical risk factors (the semicircles), one for first data entry and one for second entry or verification. These show the difference between the number of CRFs at one stage and the number at the next stage in terms of the number of days' work this represents. If a backlog seems to be accumulating, the needle is in the yellow or middle portion of the semicircle object. If there is definitely a backlog, then the needle is in the orange portion. FIG 2 - THE BACKLOG MENU
DATA ENTRY BACKLOG (DAYS)
906
If the user clicks on the critical success factor, a new screen appears.
THE BACKLOG REPORT FIG 3 - THE BACKLOG REPORT
This report shows details of the project. Each study is shown on one line. Studies that are potentially late are showri in yellow and those that are definitely backlogged are shown in orange. This report is modelled on the SASIEIS® variance report. The report may be printed or a graph showing the proportion of work represented by each study may be drawn by clicking on the appropriate button. As in theSAS/EIS® system, a notes button is provided so that the user may make his own notes for future reference. The number of outstanding CRFs for a particular study and the time required to enter them may be obtained by clicking on the individual study.
KEY FIGURES FIG 4 -KEY FIGURES
DATA MANAGEMENT TRACKING SYSTEM
The information needed to calculate the amount of work remaining is obtained from the
907
f
,
•
"
.:.;
:.':_.
~
•
'Key Figures' module. The user selects the project he is interested in or chooses the option 'All'. The figures for this project are then calculated and displayed. The average number of pages in a CRF is calculated, the last date on which data were entered is shown together with the number of days of data entry already spent on the project. Finally the mean rate of entry per day and the mean rate per operator are shown. This option is particularly useful to detect a fall in data entry rate over time as the rate will be examined daily for a critical project. The advance warning of a problem is often enough to solve or considerably alleviate-it. Problems likely to be reflected here include a fall in CRF quality and problems of response time or downtime affecting the machine used for data entry.
RATES Problems affecting individual operators are more easily seen using the 'Rates' option. This feature gives a graphical representation of the performance of each operator on each of their projects over time and a comparison of their performance on a given project.
FIG 5 - DATA ENTRY OPERATORS
TRACKING
SYSTE~f
- DATA ENTRY AWARDS
A graph showing data entry rate for a given project over time can also be obtained. This feature has been used to select temporary staff and to decide which temps should be offered full-time employment when positions became vacant.
DATA PREPARATION The modules discussed above require a great deal of memory and were taking longer and longer to run as more and more data were collected. With the second version of the system, we therefore included a data preparation module. This is used to download and summarize data from the files on the central Vax.
908
The data on the Vax are stored in 2 files, the active logfile and the historical file. This is done to accelerate the data entry part.ofthe system. Together the two tiles contain over 6 MB of information. The data preparation program summarizes this, keeping means and totals only and this summarized data forms the basis for the calculations performed in the rest of the application. To have up-to-date data available, it should be run once a day. It takes about 5 minutes on a 66 Mhz 486 PC and this is time well spent because the rest of the system is then virtually instantaneous. The summarized data are stored on the user's PC.
EXTERNAL DATA ENTRY This option is new and permits the user to indicate which studies are contracted out and should be followed up separately. Currently this just ensures that data concerning these studies do not form part of the backlog calculations. A module is currently under development to keep track of these studies and their critical dates.
SIMULATIONS A subset of the options for use by Data Managers responsible for a project is under development so that they can monitor the progress of their studies through data entry. This module will include a very popular offshoot from the main system which shows projected end dates for CRF reception and data entry based on the latest figures. Other graphs used at Project Planning meetings with senior management will also be incorporated. These show whether additional resources are needed and provide instant summaries of the current status of a project within the Department.
FIG 6 - GRAPH SHOWING PROJECT STATUS
FUTURE PLANS The most interesting of the new features planned include the addition of a module to monitor data quality and a link to the corporate tracking system.
909
The tracking system presented here is currently used at corporate headquarters in France and the US. It is planned to integrate it with the new corporate project planning system as soori as the latter is in widespread use. The tracking system will then be available for subsidiaries as well. Currently the system measures the amount of data entered in the clinical database but keeps no track of the quality. The final quality of the data depends on two factors, the initial quality of the CRFs and the quality of data entry. We plan to integrate the tracking system with our edit check system and the Quality Index*. The edit-check system performs a series of checks on the data to detect inconsistencies and anomalies. Critical factors from this system will be integrated with the tracking to· monitor CRF data quality. The Quality Index is a system developed by one of my colleagues, C. VIDY, to determine the final data entry error rate. This system is being presented at this meeting. We expect to obtain improvements in both quality and speed of edit-checking using this new tool. CONCLUSION This tracking system has taken a lot of the headache away from planning data entry. The objective evidence it provides as to our need for resources is readily accepted by senior management and we have gained a reputation for being able to provide instant and accurate . information on project status. Integration with the corporate project planning system should further refine this tool and expand our ability both to look into the future and to perform realistic projections over longer time intervals. The system has enabled us to streamline the data entry function by permitting us to detect potential problems at an early stage and to reduce their impact to a minimum. With the introduction of the Quality module, these benefits will be obtained throughout the life of a project in Clinical Data Management.
Trademarks SAS/AF and SASIEIS are registered trademarks of SAS Institute Inc., Cary, NC, USA. Decnet and Vax are registered trademarks of Digital Equipment Corporation. References * Quality Index On A Database, C. VIDY and E. OTUN, SEUGI 1994.
\
\,
910
