Best Practices for Conducting Clinical Trial EDC User Acceptance Testing

User Acceptance Testing (UAT) is a critical component of clinical trial development using Electronic Data Capture (EDC). It’s the responsibility of the Sponsor/CRO to perform UAT prior to using the EDC to collect data in accordance with a protocol.

Early and thorough UAT can result in studies that are easy to use and monitor, and have short submission preparation timelines. We’ve found this to be especially true when our best practices for conducting UAT are followed. Our VISION EDC provides tools and features that facilitate UAT every step of the way, making it a faster and more effective process. Prelude Dynamics’ seven best practices are described in more detail below. Perhaps you have some of your own you can share with us!

  1. Thoroughly test every e-CRF
  2. Verify masking and role-dependent access
  3. Involve a site
  4. Test the electronic lab interface
  5. Consider risk factors
  6. Involve the statistician early
  7. Conduct Pre-UAT

Thoroughly test every e-CRF

This may seem obvious, but it’s surprising how often this is not done. Sometimes test personnel become complacent when most e-CRFs work as expected, and abandon test efforts early. But then during study conduct it turns out that the Investigator Signoff form – which is the typically the last subject form to be filled out – isn’t working correctly.

It’s especially important to verify that the Inclusion/Exclusion e-CRF is working correctly. Imagine the consequences to the clinical trial if subjects that should be excluded are not. Similarly, dosing calculations should be rigorously vetted.

This is a good time to check the upper and lower limits on data entry fields and to make sure all calculations are operating as intended. VISION’s embedded Field Specification displays this information for every form in the study, which facilitates this process.

Verify Masking and Role-dependent Access

There’s almost nothing worse than unmasking a study, so don’t risk it! Identify every e-CRF where the treatment group (or other masked field) is present and verify that masked roles cannot see the value in either the e-CRF or the audit trail. VISION’s Field Specification is also useful for performing this task.

Inversely, imagine that the treatment dispenser has no access to the Dosing form. Avoiding mid-study changes to permissions reduces the probability of non-contemporaneous entry of critical study data. VISION’s Study Requirements Document (SRD) Configuration can be used to verify that permissions to view, edit, review and finalize each study form have been configured correctly.

Involve a Site

Sites are ultimately one of the most critical components of any successful clinical trial. If e-CRF layouts are clumsy or non-intuitive, sites will have trouble completing them correctly. When this is the case, site enrollment may suffer due to user frustration with the EDC.

Involving one or more trusted sites early in the UAT process can provide early insight into usability issues and allow them to be corrected before the study goes live. VISION’s Feedback function can be useful in capturing site input on usability issues. Happy sites require less monitor intervention and produce cleaner data.

Test the Electronic Lab Interface

Studies that benefit from the use of electronic lab interfaces, such as those supported by IDEXX and Marshfield, enjoy more accurate and timely lab results data in their study. It’s important to coordinate with lab company personnel to assure that data required by their LIMS is included in electronic lab requests, and in the e-CRFs if necessary. It’s typical to configure study-specific parameters used for tracking and billing in the EDC, and to test the interface in a training study (or training site) prior to going live. VISION supports fully validated electronic interfaces for both IDEXX and Marshfield Labs.

Consider Risk Factors

Every study has risks, but for every risk there is a way for the EDC to assist in detecting when a risk is becoming a reality. Many EDCs support ad-hoc query functions that generate reports based on user-entered criteria.  These can be configured to generate reports pertaining to potential risks such as a high number of reactions at a drug injection site, or abnormally high blood glucose level after dosing.

Losing subjects due to lack of adequate follow up poses both a financial and a schedule risk. Tools such as the VISION Event Calendar provide ways to quickly and easily detect missed visits before a protocol deviation occurs. EDC statistics functions can also be used to view data in graphical format, which is more intuitive than rows of numbers and can be used to identify outliers.

Custom reports can be developed in the EDC that are tailored to look for specific potentially risky outcomes. These can be developed at study build time, or added later when unanticipated risks begin to arise during study conduct.

Involve the Statistician Early

Configuring the study database so that the data is organized in a way that makes it easy for the statistician to prepare the data for submission is something that is sometimes overlooked when studies are being developed. This can result in both time delays in submission and difficulty with data analysis.

We recommend entering test data into either a training study or training site, and exporting the data for analysis as part of UAT. This approach allows the statistician to take an early look at the exported data, and test it with their statistical analysis software. If changes are needed to the database to facilitate analysis, these can be made and testing prior to going live.

Conduct Pre-UAT

Pre-UAT? It’s not unusual to identify multiple study changes while testing, especially when unfamiliar with the power and flexibility of EDC. Conducting a Pre-UAT ahead of the UAT schedule can help identify and address most of these changes ahead of time. When this is done, UAT consists of verifying that all requested changes have been implemented, tested and work together in an intuitive way.

Starting study test early is generally recommended to avoid cramming test efforts into the last week prior to going live. When testing is deferred, sites have a more difficult time entering data and study conduct is more difficult and lengthy than it needs to be.

Conclusion

In our experience, following these best practices for performing UAT results in less risky, higher quality studies with excellent usability and data integrity. EDCs such as VISION provide many validated, integrated tools that facilitate both UAT and study conduct. Prelude Dynamics provides training in the use of these tools prior to the conduct of Pre-UAT and/or UAT. We are increasing receiving requests to deliver the data from training studies/sites as part of study closeout. This data serves to demonstrate that UAT has been conducted.

If you’d like more information about any of the tools and features discussed in this article, or would like to schedule a demo, please contact us.

The Future of Visual Study Archives

Your EDC study has come to a close. All data has been finalized and the database has been locked, and now –it’s time to generate your study archive. How do you make sense of all that data visually? The last thing you or your data reviewers want to do is struggle to make sense of overwhelming data delivered in an endless and flat-lined, row-by-row spreadsheet.

In Today’s results-driven landscape, data aesthetics can often play a back-seat role. Large XML files storing computer-strung data points are comprehensive but not entirely human intuitive. Data reviewers can easily overlook relevant data while struggling to understand the visual context in which that data was entered. Flat XML files provide the full data in a raw form, but the visual picture of the study’s conductive methods is incomplete without…. well, visuals of the CRFs. What are the current solutions around this data presentation problem?

In recent years, visual PDF copies of case report forms have been the answer, but with PDFs there is still the problem of how to manage and bookmark every single screen of the study that was once viewed online, in a web-based EDC system. The PDFs are stagnant screenshots and require scrolling through a library of them, in a tiresome fashion. The ability to scroll through and navigate among every single PDF, paralleling the method in which the data was entered is just not feasible.

Prelude Dynamics has a better solution: an offline HTML archive. Dynamic and intuitive in structure it is easy to generate, access, and use. an HTML archive means no longer becoming entangled in a web of spreadsheet columns, unmanageable libraries of PDFs, and no more costly server fees to perpetually host the study for intermittent visual access. The HTML archive displays the entire study and its layout for you with a visual presentation of all data and function captured in one place.

The beauty of an offline HTML archive lies in its dynamic use and its cost reduction; all the appropriate links, highlights and audit trail per form are preserved and navigable exactly as they were presented within the EDC upon collection, but without the cost and security concerns of keeping a server running indefinitely. An offline HTML archive captures all the same data and their hierarchies in a read-only format just as it appeared within the EDC. This makes it infinitely easier for the auditor or data reviewer to navigate and visually scrutinize data results in question. Who entered what where and why are frequent data review questions easily answered with a visual archive.

With Prelude Dynamics’ VISION™ solutions, clients can receive the offline HTML archive in addition to a fully provided XML database (a SAS-approved import format and a native format.) The HTML archive really shines in providing a visual glimpse of studies past in read-only fashion, without altering the data format and results which were initially entered. HTML pages remain locked so they are not editable or alterable when viewing through any browser.

As clients grow the scope of their studies to get the most out of EDC, Prelude Dynamics provides greater solutions for data organization and presentation, in line with the expanding data submission requirements from regulatory agencies world-wide. Regulatory groups are putting increasing emphasis on the review process, and we believe that an offline HTML archive solution will surpass the PDF standard in greater visual accuracy, navigation, and preservation of data.

The future of visual study archives is offline HTML, an emerging standard that makes archives, data review and submission simpler and more fluid for all stakeholders and sponsors, and effectively expedites and refines the review of clinically studied products and their efficacy. HTML archive is an archive solution worth the investment.

VISION™ Dashboard: The Future of Study Monitoring is Here

With the newest advancement of VISION’s Dashboard tool, monitoring your study or trial has never been easier or more adaptable! With more monitoring capability located in one place, you have the power to review what you need at a glance and dive deeper for further review.

Gone are the days of scattered summaries and reports. With the updated Dashboard you can now monitor what you want, when you want and all on one page with data summaries that you control. At Prelude Dynamics, we understand the importance of quick, convenient monitoring that you don’t have to dig for and in this article we’ll showcase some enhancements to the Dashboard that give you exactly that!

Similar to a news feed, VISION’s Dashboard enables you to decide what data criteria you want to see. No matter what your data monitoring preferences are, we give you full control to arrange and filter your data within the dashboard to your needs while also including fundamental tracking tools like a Quick Stat overview, messaging module, and a built-in study calendar. Real-time data reports and summaries are now all conveniently located in one place – the VISION™ Dashboard.

To access the Dashboard, simply click the dedicated Dashboard icon in the main navigation while logged into VISION™:

When you first arrive you’ll be prompted to take an automated tour of the new dashboard features. One new feature is that your dashboard is fully configurable to show you the data that you need to monitor. You can choose from a variety of summaries, graphs and charts, display the ones you want, hide those you don’t and sort them in any order. Just click the +/- icon to show a list of active and disabled summary modules. Any module you want to view, simply drag it into the “Active” column, click Save and you’re done! Your dashboard is customized and you are ready to monitor.

The Dashboard has 3 Main sections: Overview, Messages and Calendar…

In the Overview section, the Quick Stats module lets you select from important monitoring data counts, like: number of adverse events, serious adverse events, monthly enrollment, number of unresolved queries or total queries, form status data and more. You can then add numerous charts and graphs showing at-a glance visual representation of various data within the study.

Within Overview, you decide which data points to monitor and in what order they are displayed. For a more detailed breakdown on any of the summaries, simply click on the label to access additional information which will display within a table or chart.

In the Messages section, notifications are conveniently displayed in a list so you can quickly review sent or pending notifications, acknowledge them or even compose new ones. The list can also be filtered by category such as adverse event, new patients enrolled, queries, protocol deviations and more to show you only what you want to see.

In the Calendar section, you get a view of the current month showing personal, site and study created events that are color-coded to help keep track of each. You can navigate to view past, present and future months, add and schedule new events and filter them by type to focus on only those events that are relevant to you to help plan and schedule accordingly.

As a Monitoring tool, the Dashboard is a comprehensive one-stop shop. You can pull in any existing subject, site or user summary or any ad-hoc report that you created. This provides you with the ability to view all the important study data in one place, and eliminates the need to click through reports and summaries in separate locations.

It gets even better! You can now also filter the dashboard by a given site or view data for all the sites as a whole. Each report and summary can be filtered separately by any custom search you create. If you want to see only data for enrolled patients, select that filter. Create new reports and filters at any time and they become immediately available for use in the dashboard. You can even pull in shared reports and searches from other users.

That’s a quick dive into our enhanced Dashboard monitoring tool available for all studies with our newest version of VISION™. At Prelude, we believe in the power of convenience, configuration, and time-saving tools for you and your team. Welcome to VISION™ Dashboard!

VISION™ Tips & Tricks: Utilizing Summaries


Did you know that each VISION™ study comes with a set of standard Summaries?  They aggregate Subject and Site-level data, enabling faster monitoring and data validation. Standard Subject-level summaries may include Subject Status, Form Status, Query Details, and Comments (Note to Field).  Standard Site-level summaries may include Site Status, Enrollment, Form Status, Queries and Comments (Note to Field). Custom summaries are also available upon request (and include, but are not limited to Adverse Events, Deviations, Note to File, etc.).

 

To access a Subject or Site Summary, navigate to the Subject or Site list page and Click the “Summaries” button in the footer of the page.
Summaries
Also, please note that if any Searches or Filters have been applied, the Summary data will only display the filtered data set. At the Subject level, you can view Summaries for all sites, or a specific site by specifying in the site dropdown (see Figure 1).

Figure 1. Site selection dropdown can be found in the page header.

You can then narrow the scope of the subjects you’re viewing by applying search criteria (see Figure 2).

Figure 2. Adding Search criteria can help narrow the scope.

See our Tips and Tricks on Layering Data Searches for more details on applying search criteria.

Below is a table of examples found in VISION™:

Summary cohortSummary dataDescription of Summary
SubjectSubject Visit StatusShows the most important dates per subject. (Study visit dates, study completion dates, etc.)
SubjectForm StatusShows the # of forms in-work, in-review, reviewed, finalized for each subject. Also shows the # of unresolved, resolved and reviewed queries per subject.
SubjectQuery Details Summary contains two sub-tabs, one for "Unresolved Queries" and one for "All Queries." Details include subject ID, form/field query was entered on, date, who entered the query, the query, and the time it took to resolve the query (or the time it has been open).
Subject and SiteNote to FieldShows an aggregate of the notes made to subject (or site) fields
SiteSite StatusShows aggregate enrollment by gender, screen failures, and completed study status. It can also display Form and Query status.
SiteEnrollmentShows a graph and/or table of expected versus actual enrollment per site.
SiteQuery GraphShows total versus unresolved queries. Excellent for a quick view of query status overall and for site comparison.

Please contact your Prelude Project Manager for more details or questions on this article.

Lessons from SCDM 2017: The changing role of Data Managers in EDC-driven studies

Prelude recently presented at the Society of Clinical Data Manager’s Annual Conference held in Orlando, Florida. Alicia Browner, Ph.D., our CTO, was joined on a speaking panel by Karen Lane, Ph.D. from John Hopkins University and Jess Wofford, DVM from Aratana Therapeutics. Together, they provided an overview of the changing role of Data Managers in a technology-infused clinical trial environment. For those that did not have the opportunity to attend, we would like to share some of the key points presented:

  • Early engagement of the data manager(s) for an EDC-implemented study is imperative – especially in the initial phases of electronic study development and design. Data managers are experts for cleaning data, quality-controlling data input and entry, and are invaluable in testing an EDC system against data capture compliance and regulations and adherence to protocol.
  • One of the roles of data managers is to become the knowledgeable advisor that can look at the EDC critically and leverage all of it’s functionality to clean data as it is entered, in real-time. This is achieved through automatic edit checks, custom calculations and algorithms. All this reduces the amount of monitoring necessary and improves the quality of the data.
  • When it comes to monitoring, the data managers are instrumental in identifying risk within the study. If risk-based monitoring will be implemented, the data managers could help identify areas in which there might be a protocol deviation, where data might not be entered contemporaneously and how that might be monitored. Data managers could find what fields should be strategically scrutinized to ensure patient safety. This includes requiring certain users to be automatically notified if certain criteria are met and how to identify fraudulent data or deal with missing data.
  • Data managers can also influence day to day operations by helping the team understand and use other EDC functionality such as inventory management, automatic reordering, invoicing, document management, and more. These automated functions can streamline operations while improving data quality dramatically – definitely a winning strategy!

During the panel, Karen Lane of John Hopkins University shared how they used the system to track performance and enrollment metrics. One graph showed the significant increase of efficiency they experienced in enrollment when they moved to EDC. They also created a friendly competition among sites to encourage enrollment and had a dashboard where sites could quickly view their ranking. The result was a dramatic improvement in enrollment.

 

Jess Wofford of Aratana Therapeutics shared their lessons learned in adopting EDC. One issue they had previously experienced was that sites used different units of measurement for weight and the subsequent human errors in converting those to a consistent unit. They solved this with EDC by enabling the sites to select only lbs or kgs and the system automatically calculating the conversions (lbs to kgs and vice versa.) While this might sound simple, it really was genius; some scales measuring in lbs read out both lbs and ounces while others provide lbs with decimal points. To account for this two text fields were provided in the EDC. The fields for lbs could be used to enter whole lbs or lbs with decimals depending on what the scale displayed, and for those scales showing ounces, the information was entered into the corresponding field. (See Figure 1) This was extremely helpful in helping accurately track the weight.
Figure 1. Body weight measurements and automatic conversions.

Aratana also took advantage of the EDC capability which hides forms once they’ve been signed off. Why is this so important? It helps to reduce bias when conducting repeated measures. During the session, Dr. Wofford also mentioned that they added a function where the monitor could make the form reappear if there was an outstanding query so the query could be answered and resolved. They thought of everything!

We greatly appreciate Aratana’s Dr. Wofford and Johns Hopkins’ Dr. Lane for sharing their experiences and providing lessons learned from their own perspective and illustrating the importance of the data manager role in EDC-driven studies. One thing is clear from the presentation: Data Managers can serve a crucial role in the development, design and risk assessment of EDC-driven clinical studies and we at Prelude look forward to working alongside DMs to deliver study data capture that is clean, accurate, risk-averse and quality-driven.

The Future of EDC is Here!

Last month we attended SCDM Annual Conference to present a panel on “Cleaner Data Faster: Leveraging EDC/CTMS in Global Studies”, we also attended a session on the “Future of EDC”.

The session began with a discussion surrounding eConsent, eSource and ePRO as the future of EDC, the next frontier of eClinical solutions. We were over the moon! We could barely contain our excitement. With Prelude and VISION, the future is already here! As many of you know, VISION has been collecting eSource documents for over a decade! Some of you have taken advantage of having patients log into our system for nearly as long. While we have not called it specifically eConsent and ePRO, the functionality of allowing patients to enter their own data, access their consent form and sign it electronically has been a refined capability of VISION for a very long time.

The session also discussed wearable devices and having the ability for an mobile device to input information directly into our system. You might already know that we have direct lab interfaces with IDEXX Labs and Marshfield Labs US and continue to work in partnership with other labs to integrate systems. VISION also seamlessly pairs with RFID and barcode readers. VISION’s technological flexibility allows us to develop an efficient interface with just about any external device or wearable and collect data necessary for your project.

Finally, they discussed advance computer learning and virtual and augmented reality. Much of the advance computer learning is still human driven, i.e. someone has to think about what is important for the computer to focus on and what the computer should do when it identifies that pattern. The reality is that humans are still figuring that out. The same goes for virtual and augmented reality, even the speakers were not sure exactly what that might look like or how it might be applied. We can assure you that we are constantly listening to what your needs and the needs of the industry are and continuously develop functionality to streamline your operations.

Most recently statistics, graphing and a dashboard were added to our VISION product line, just to name a few. These are updated instantaneously as new data is available. We feel that this is the first step towards providing you with advanced computer learning and currently all that can be done until someone is able to verbalize what else is needed; the panel did not provide details.

In conclusion, the future of EDC is here. It is called VISION. It is in your hands. Whether you contract our Premuim package or choose to use our build it yourself Composer tool, you have access today to the future of EDC and it’s never looked so bright!