Study Monitors have a lot on their plates. While VISION’s automated data cleaning capabilities make their jobs easier, we thought it be useful to provide them with a tool that would automatically flag database fields that are more “challenging” for sites to complete. Our new AI Monitoring Enhancement Engine (AIMEE) does just that. When monitors are reviewing a form, they can turn AIMEE on with a single click of the mouse and instantly focus on database fields that are historically known to require attention.
A single click of an icon turns on AIMEE.
AIMEE calculates a score for each field and highlights those that require attention.
How does it work? AIMEE learns over time and calculates a risk score for each field on the form, and highlights those that merit further scrutiny. What is this score based on? AIMEE looks across all instances of the form in the study and considers the number of changes, queries, comments, and occurrences of errors and warnings for each field on the form. It uses that information to generate a score, which is intuitively displayed next to each field on the eCRF. Low scores are “good” and high scores are “concerning”.
AIMEE works well in conjunction with VISION’s field-by-field review functionality – which monitors can use to indicate which fields they have already investigated. This in turn lowers monitor workload by helping them avoid duplicate efforts.
Reach out and ask for an AIMEE demo today!
Schedule a Demo
What does it mean to conduct effective remote clinical monitoring? It’s critical to be using an EDC in order to accomplish this, but what else is necessary? Keeping the primary goals of the monitor in mind, we will outline several techniques for streamlining this process.
What are the primary goals of the study monitor? First, and most critical, is to ensure the safety of all study participants – including clinical personnel. Second, is ensuring data quality and integrity.
Let’s discuss the EDC features that can help in each of these areas.
Monitors are relied upon to use their knowledge and expertise to recognize potential safety risks, and to report them when discovered. The EDC can’t replace monitors, but it can assist them greatly in their daily workflow.
Use of Notifications. The EDC can be configured to automatically notify monitors when certain events occur. Examples of this include the entry of adverse events, the reporting of out-of–range lab results and new subject enrollment. When automatic notifications are used, monitors are informed in real time and the need to continually review eCRFs for potential issues is eliminated – saving valuable time. Think about the potential risk factors for a study and consider how notifications can be configured to assist the monitor.
Use of Reports and Graphs. Some EDCs support flexible, configurable reporting tools that facilitate the aggregation of data for review. Graphs can be used to more intuitively spot outliers and aid in understanding trends. If such tools are not available, consider asking your EDC provider for custom summary reports that can be used to review study endpoints, exam results and adverse events in a comprehensive way.
Use of Study Dashboards. Dashboards that aggregate reports from multiple sources can provide a convenient starting point for a monitor’s daily workflow. Monitors can log in to the EDC and navigate to a central page to review notifications, study metrics and risk-aware reports.
By proactively thinking about the types of information monitors need, each study can be configured to provide tools such as these before the first subject is enrolled. With the VISION™ EDC system, mid-study updates can easily be made to support data monitoring needs that weren’t anticipated at study build time.
Ensuring Data Quality and Integrity
Sophisticated EDC systems support automated data cleaning in a variety of ways, which naturally improves data quality. Monitors and data managers can contribute their expertise at study build time to make sure that this automation is included as comprehensively as possible.
Use of “Completeness” Checks. Completeness checks answer the question “is all required data on the eCRF?”. This is especially important when some data can be entered conditionally (e.g., only if another data entry element has a certain value). A related question is “why wasn’t this form been filled out?”. It’s a good practice to implement a “Not Done” checkbox on forms such as Adverse Events and Necropsy that are (hopefully) not often used. This practice lets monitors know in a proactive way that the form is intentionally blank, which effectively reduces the amount of questions from the monitor.
Use of “Correctness” Checks. Correctness checks help answer the question “is the data correct and in the right format?”. While study protocols usually provide information about the resolution (e.g., number of decimal places) of data to be collected, they don’t always specify things such as the expected range for data points. Enlisting the support of data managers at study build time will help ensure these checks are in place and are as complete as possible.
Use of Visual Error Indicators. When visual error clues are present and easily viewable, monitors don’t have to spend time manually reviewing each form for errors. They can focus their efforts on strategic reviews instead. Such indicators also provide immediate feedback when data is entered, which prompts clinical personnel to correct entries in real time.
Use of Document Uploads. When it is necessary to transcribe data from paper forms, it’s good practice to provide an upload mechanism for those forms so the transcription can be reviewed more easily. This mechanism, along with Source Data Verification (SDV) functionality, is also useful in converting a paper study to EDC. VISION’s SDV functionality allows the monitor to sign off on each data entry field individually or at the form level.
Use of Integrated Query Functionality. Having the ability to enter queries directly into the EDC system, and to easily see whether queries have been resolved, is an enormous aid to monitors. It eliminates the need for them to revisit queries that have been resolved and move on to more important work.
Qualify the EDC Vendor’s Infrastructure. We’ve defined mechanisms for ensuring the completeness and correctness of data, but how secure is it? EDC vendors should be asked about their backup strategy. How often is data backed up? How much data could I potentially lose? How long does it take to restore data? How long is data retained? These are some good questions to ask your EDC provider before your study commences. Prelude recommends (and performs) hourly backups. Questions about barriers to system access are also important. Ask your vendor what their average up-time is. If clinical personnel can’t enter data in real time, data integrity could be impacted.
Monitor Workflow – Continuous Monitoring
Here are some recommendations for conducting remote clinical monitoring using VISION™.
- Identify a monitoring schedule and stick to it. For slow-enrolling studies, this might be once or twice a week. Larger, more active, studies may require full time monitoring efforts.
- Create focused reports that highlight potential risk areas and promote identification of outliers or undesired patterns. Configure the Dashboard to include these reports and share them with other clinical personnel to promote awareness.
- Consult the Dashboard to make sure notifications have been handled and to get an idea of emerging trends that require analysis.
- Review forms that have been submitted by clinical personnel and use the Query function to call attention to questionable data.
- Review recent lab results for out-of-range results and determine if those relate to Adverse Events.
- Review fields with changes to see if there are any patterns and clarify their use with clinical personnel.
It’s important to perform these activities continuously. Contemporaneous identification of issues is critical when safety is concerned and helps clinical personnel recall specifics they are likely to forget after time passes. Also, when monitoring is done continuously, study close-out activities can be completed much earlier since most of the work has already been done!
Monitors who understand the capabilities of their EDC provider and the automation that has been included in their study have an advantage. They can rely on the EDC system to handle the “grunt work” and concentrate instead on critical outcomes and analyses that can’t be automated.
There are so many ways to design electronic CRFs. Perhaps by looking at how NOT to design them, we can come up with some fresh ideas for how to make your next study a smashing success.
Fail to consider how well the forms will function on handheld devices.
The sun us setting on big screens and desktop workstations. Soon a new day will dawn, and the sun will shine solely on small screens. They provide mobility and greater access across the board. They are the key to conducting business in the information age.
Take, for example, form layout. Regardless of whether you are viewing it in landscape or portrait, a long list of fields is not conducive to the thought-processes of the human brain. Fields need to be grouped together in a way that makes sense to the person filling it out. And news flash: the person filling out the forms for your study is guaranteed to be relying on a human brain.
Put simply: don’t shortchange the design process of the study build in favor of a quick turnaround. You will thank yourself later when you are parsing through the raw data for usable statistics.
Require users to navigate extensively to do their jobs.
Do not bury important fields at the bottom of the form. Don’t force your sites to have to dig deep to get what you need. Cater the form configurations to quick and easy navigation.
Employ checkboxes instead of radios.
Simplify the investigators’ jobs before they even begin. Give them less options when possible. Dummy-proof your study to the point where the patients themselves could fill out their forms.
Don’t provide options for “Neither,” “N/A,” “Not Done,” or “Other.”
Bottlenecks happen when a field is lacking necessary options. The investigator will simply leave it blank if they do not see the option they need. Then the monitor goes in and queries the blank field. The investigator responds that they didn’t perform that particular test. The issue gets shunted up the line until it gets to Prelude and a mid-study update is requested to add the pertinent option to the field in question. This takes time, causes confusion, and costs extra support hours, depending on the complexity of the request. There will always be mid-study updates. It’s a normal part of the process, but some things can be prevented. Do your best to predict issues before they happen, and your study will run much more efficiently. Just don’t overthink it…
Assume that the data will be entered in a certain order.
After taking into account all of the above, we’ll leave you with this: You can’t predict everything. Put as much thought into your eCRF’s as you can, but first you have to learn the difference between what you can control and what you cannot. Providing a radio in place of a list of checkboxes is all well and god, but don’t assume they will fill out the first item on the form first. Don’t go overboard with field suppressions and calculations. Try to think the way a clinical would. That’s all you can do.
Streamline data-capture by using Vision’s built-in documents manager. You can keep all of your data and communications in one, easy-to-navigate place. Not only will assigned users have instant access to communications and updates, but you will be able to track who has viewed or edited what. No more unresponsive team members claiming, “No, I didn’t see that last update!” when really, they just forgot to respond to it (you know who you are).
Simply navigate to the applicable patient or site where you intend to upload a file or attach a note. Typically, the last or second-to-last form under each patient and site is labelled Notes/Comments/Errors or Comments/Errors. Under this file group will be a notes form. This can be used to append documents like medical histories or previous labs results to a specific patient or signed forms to a specific site. Team members can also add short comments as necessary throughout the study. Prelude offers this form group as part of every contract, but it can also be customized to suit your unique study’s needs.
There is also a form where users can upload transcribed CRF’s, making it easier than ever for sites to keep a thorough record of all essential documentation for their subjects. This basic form includes a place to record the date and time uploaded, the name of the electronic version of the form where the data was transcribed, as well as a link to that form within the system, the reason for transcription, and checkbox where the monitor can indicate that they reviewed the transcription. But don’t forget: this form can be customized to include any extra details your study requires.
Ask your Prelude Project Manager about even more ways to tailor your study’s forms to meet all your document management needs!
An EDC improves efficiency and reliability while conducting clinical research. Here are some ways it does that:
1. Efficient Communication
The efficiency of communication between team members is what determines the success or failure of a clinical trial. If a team does not have a streamlined process for disseminating information throughout the life of a project that deals with the staggering amount of data typically generated by a drug trial, everything else can fall apart. An Electronic Data Capture (EDC) system provides a platform for making sure information gets to the proper people in a timely manner in a way that paper will never be able to match.
With an EDC, there is a much lower risk of communications being lost or seen by the wrong people. Updates can be shared instantly to ensure that everyone who needs to know will have the pertinent information at their fingertips with no waiting. The only thing you will have to wait for is federal approval (sorry, can’t help you there).
2. Instant Documentation
Every time a user changes a field or makes an entry into the EDC system, the time, date, and user information is captured and recorded instantaneously. Indeed, as soon as a user logs onto the study, the EDC system records how much time they spent on each form even if they simply left their browser open while they played Solitaire in another window (disclaimer: we do not, as yet, have a way to track for you who on your team is playing games and who is actually working).
The EDC system can also record signatures on forms along with accurate date/time stamps. Say an investigator forgot to sign off on a form, and it goes unnoticed until the end of the trial. With paper, the monitor would have to track down who the investigator at that site was and mail them the form to sign. Then the offending investigator would have to mail the signed form back to the Monitor. If the investigator forgot to sign the form initially, who is to say they won’t also forget to send the signed form back? This process is time-consuming and susceptible to errors. With electronic data-capture, signatures are a given and no one has to wait by the mailbox for clerical corrections.
3. Intuitive Summaries
Most clinical trial personnel would agree the most time-consuming aspect of a drug trial is the extrapolation of useful information from the massive amounts of data generated throughout any given study. Where paper fails, an EDC system thrives. The EDC platform is designed specifically to make combing through data as efficient as possible (we won’t go so far as to say it’s enjoyable, but efficient, definitely). An EDC accomplishes this by providing user-friendly summaries directly within the system, which saves time and energy by doing all the work of combing through patient data for you.
Summaries can be set up to meet almost any parameters you can think of, and they can be set up in a matter of seconds. You can compare infinite pieces of data (provided you have infinite pieces of data) to help you come up with the most accurate results possible for your study without wasting man-hours (or woman-hours). Most importantly, EDC summaries can be saved, altered, and shared as fast as thought, so you can keep up with the continuously evolving business of producing safe drugs for people and animals.
4. Real-time monitoring
Monitors can review data entered into an EDC System as soon as the person entering the data hits ‘Save’. There are even features which allow monitors to view data-entry live from a remote location to ensure that patients are kept safe during difficult or potentially dangerous procedures. An EDC system keeps a study moving forward by removing lag-time between clinical observations of patients and safety monitoring of documentation.
5. Error Minimization
The biggest shortcoming with recording data on paper might be the amount of errors that go undetected. When using an EDC system, where every field utilizes automatic constraints and calculations, often in tandem, errors are not only much easier to detect but they are also much easier to prevent.
Constraints can be placed on fields to restrict an impossible value from ever being entered in the first place. If there are unforeseen types of errors that the constraints do not account for, then the monitors can easily attach a query directly onto the field in question. Then the system can be updated to catch any future errors of the same kind. More levels of validation lead to less mistakes.
6. Customizable Forms
Every clinical trial is unique, and, therefore, every clinical trial has unique requirements when it comes to forms and fields. An EDC system, being highly customizable, offers the ability to only show the fields necessary to be filled out for each patient and visit. This saves time for the investigators when recording their observations. They do not need to waste time filling out an entire stock form as they would on paper.
Forms in an EDC system can be set up so that the next field or form becomes visible or remains suppressed depending on the answer to a previous field or form. In this way, the system guides the data entry personnel through the appropriate process and prevents a lot of headaches. Plus, not having papers means no paperwork!
One of the most important aspects of the clinical trial process is making sure that the right people have the right access to view and edit study data. One could argue that the success or failure of a drug trial hinges on the proper assignation of permissions to each team member. Permissions are the barriers that prevent important data from falling into the wrong hands or, equally as problematic, not being received by the right hands. The key to managing the user permissions for your trial is understanding what the basic user permissions are. From there you can customize them depending upon the needs of your unique trial, with help from your EDC provider.
The first and most important thing to note before you begin assigning permissions is that one, and only one, member from your team should be assigned to manage all user permissions. This is typically the job of your Project Manager (PM), and it is important for that person to be trained on permissions, in general, and your team’s needs, specifically. Your Prelude PM can help you set up the initial users if you are unsure where to start, and they can offer guidance and support throughout the life of the trial. It is up to you and your team, however, to ensure that you have the necessary users set up with the proper permissions before your trial begins. There is no one set process for determining your user permissions, but it is important that no more than one person should have the ultimate power to edit and manage the users and permissions in the system. This measure is in place to prevent conflicting user assignments or improper access to data being given to users. Having a streamlined process for managing user permissions is a vital pillar of Good Clinical Practice (GCP). It also mitigates confusion and promotes communication.
The second thing to note is that the earlier you get your users and permissions set up, the better. The farther into a study you get, the more complicated and potentially problematic it becomes to change the permissions. For example, if you determine that the Investigator should not have been able to view the treatment assignments of their patients halfway through the study, your results from the first half of the study might be deemed unusable. That would be bad!
Finally, do not rely on your EDC provider to know what permissions you need for your trial. It’s your study and your neck on the FDA chopping block! This is where proper training comes in. Here are four basic user-access categories to keep in mind when setting up and managing your user permissions and remember: your Prelude team is here to help and guide you should you need further customizations.
1. Prelude personnel
This includes your Prelude PM and Prelude Quality Assurance (QA) team. The PM will have Administrator access-only to your Live study so that they can assist you with user access issues and mid-study updates. QA will also be able view all study forms so they can ensure they meet industry standards. Neither of these roles will be given data-entry access to patient forms.
2. Non data-entry personnel – This category includes two sub-categories, both of which are given view-only access to patient forms:
- Roles which can view and edit all user/site/project forms, e.g., Project Manager, Monitor.
- Roles which have limited access to view and edit user/site/project forms, e.g., Coordinator, Medical Monitor.
3. Data-entry personnel – This category includes two sub-categories:
- Roles which can view and edit all patient data, e.g., Investigator
- Roles which have limited access to view and edit patient data, e.g., Patient, Lab Technician.
4. View-only personnel
These are the roles which, like Prelude personnel, cannot edit any data and can only view the forms which are necessary for them to view (again, as determined by you and your team), e.g., Sponsor, Statistician, Data Manager.
Of course, there will always be exceptions to these basic permissions. Perhaps your Medical Monitor should be able to enter patient data, or your Lab Technicians shouldn’t be able to view site data. You can also, of course, add or remove roles according to your every whim (well, maybe not every whim, but we’ll do our best). This is the beauty of using a highly customizable system like the one provided by Prelude. The possibilities are endless! Whatever your study needs, Prelude delivers. Our system is designed to make your study run as efficiently as possible by catering to your unique needs. Customize away!