A Case for HTML Archives for FDA Submission

Currently FDA official submissions must be in XML or PDF format, with the preference being the submission of at least the XML format. The rationale for these is the need to have a document that will be readable and accessible in perpetuity. This is definitely an important consideration. However, these formats contain limitations when it comes to the review, oversight and approval process as clinical reviewers are more effective “seeing” a case as a set of forms the way the Investigator filled out those forms, while analysis staff are more effective with the XML raw information. How do you handle ancillary data, such as images and graphical information that were part of the process which cannot be imbedded in XML?

Below we will present a case for allowing HTML archives to be added as a part of or one of the allowable formats for FDA submission, fundamentally to support clinical case review. The raw XML would continue to be favored format for data analysis.

HTML, Hyper Text Markup Language, is the foundation of all computer languages used to create web-based interfaces. HTML is a specialized form of XML that when combined with images and style sheets (CSS) make a web page appear as it does. The Internet has been growing exponentially since its inception transforming business and everyday life; the Internet is here to stay and so is HTML. “Billions and billions” of HTML web pages exist in the world today. HTML can be compared to the alphabet. Like the alphabet, it provides us with basic element in which we can combine in order to communicate via web pages. HTML does for computer language what the alphabet does for our language. It is safe to say HTML will stand the test of time.

One might assume that HTML requires a web server to operate, but in fact, an HTML page is just a document that resides on a computer hard drive on a server, and can also be saved locally. All browsers support “Save As” and can save a web page for later viewing when the computer is disconnected from the Internet. All that’s needed is any brand of web-browser. As such, these HTML/XML pages are safe, secure, and as perpetual as any computer file on the hard drive.

Now that we have established that HTML is a viable solution in perpetuity, let’s examine the differences between XML, PDF and HTML archives. The main differences are listed in the table below.

 XML ArchivePDF ArchiveHTML Archive
Accessing softwareXML can be imported into many database software programs as well as some spreadsheet programs. It is truly non-proprietary, and readable by humans with just a text viewer.PDFs can be read using a number of portable file document readers. The PDF format has been made public, but is proprietary and licensed by Adobe.HTML is truly non-proprietary, and can be read like XML, but looks best when combined with CSS in any browser.
Security and Virus vulnerabilityNo vulnerability.No current vulnerability but PDF enhancements may someday add scripting.HTML can have embedded JavaScript that a scanner would need to remove. These same malware detection components are included in all modern browsers.
Availability of software to access archive typeMost of these have to be purchased.Some of the readers are available free, but the population in general does not all have them.All computer users have access to free browsers.
NavigationXML is typically hierarchical which reduces duplication compared to repeating table rows. Navigation is by drilling into the hierarchy much like opening folders with subfolders.Data is presented in lengthy documents that are difficult to navigate, but that display the forms upon which data was entered. Limited capabilities to search and find information.Displays the data in the forms that they were entered in providing the viewer an experience similar to what was used when the data was input. The viewer is able to navigate through pages using the same process the data collector did.
Handling of uploaded documentsNot contained within the XML file.Usually available as separate PDFs in a separate folder with no link back to the patient or form.Documents are accessible through clicking the embedded hyperlink. Documents are in their original format or PDF.
Visibility of queries, field notations, and changes to data Prelude’s XML has queries, comments, and changes co-located with the field variable to which they apply. No separate files. However, XML while a simple structure and open format, is verbose and reviewers are challenged to view a “case” as an entity.The PDF as a screen image will show what was on the screen, but may also be a separate page or even a separate document. This can make searching and finding difficult.HTML Archives are saved web pages, and when done properly, allow users to click the hyperlinks to view and navigate the specific query, field notation or change to data related to a given field at the point of viewing the field.
Overall functionality for analysis (e.g. SAS)XML provides data that can be freely processed by many non-proprietary systems to support a data analysis.Does not allow data processing.Does not allow data processing.
Overall functionality for case reviewMost FDA personnel who review and provide study oversight are not data managers or statisticians; they are monitoring personnel who are more familiar with a paper-based analog (CRFs) process where they can look at the forms and see the data being collected in context as the Investigator staff viewed the case. XML does not provide a format that is easy for them to use.It is difficult to navigate through long PDFs of patient case report forms and to navigate between patients is nearly impossible or requires opening multiple individual patient PDFs.Provides excellent functionality allowing navigation from patient to patient and form to form in the same fashion that a person entering data would have done so. It makes the process of reviewing and providing oversight much faster because they do not have to organize the data and can focus on reviewing the data to ensure the safety and efficacy of drugs.

In conclusion, while the XML archive is a natural non-proprietary requirement for data submission because it does allow quick processing of the data in order to analyze the safety and efficacy of a drug, providing HTML data in addition is superior to PDF files in that it facilitates the case review within the same context in which the data was originally collected by Investigators. HTML will increase FDA reviewer’s confidence in the quality of the data and its outcomes.

Prelude Dynamics has initiated conversations with FDA CVM to discuss allowing HTML archives to be submitted. Among the discussion is also the possibility that Sponsors could chose to allow CVM to have access to the server in read-only mode where they could interact with the study data and even run reports, statistics and graphs within the system before final submission occurs in order to guide sponsors on any additional requirements or concerns they might have in order to help expedite the final review process, reduce the need for clarifications, and be quicker to market.

Contact

Jim Pedzinski – VP of Business Development
jpedzinski[at]preludedynamics.com – 512-476-5100 ext. 210

Reducing Qualitative Data Analysis Through Autocomplete Text Fields

Background

One of the difficulties in doing research that occurs during the data analysis phase is how to analyze open text entries. Anytime data is collected into a text field, it becomes a qualitative data point when it comes to data analysis. Analyzing qualitative data tends to be more time consuming and is open to the analyst’s interpretation.

In some cases it is difficult to come up with all the options to include in a drop down choice field. So many times, other is offered as a selection and a text entry field is made available to capture the other descriptor. In many cases, a text field is offered simply because there might be a lot of other type options even when most people will enter the same response because the potential answers are not part of a validated scale. Whatever the reason, this makes for complex qualitative data analysis.

Challenge

The challenge was to develop a way to allow both free text entry and a choice of drop down items. Ideally where the drop down list grew as a result of prior entries into the field and where the choices would appear once someone began to type into the field. That way the person had a choice to select an option or continue typing.

Proposal

Prelude Dynamics proposed that it could add functionality to VISION™, an electronic data collection system, that would capture the information entered into the field and display it back as selectable options within a dropdown list.

Solution

Prelude added functionality to VISION called an autocomplete function. The autocomplete captures information that is entered into a text field and when the next person starts to enter information, offers the previously entered responses. In this fashion, the subsequent users have access to a growing list of options to choose from. With a little training, users would know to choose the best option that describes what they would enter or add a new option where none fit.

Results & Benefits

Autocomplete has allowed data entry of qualitative fields to become organically quantitative. As information is entered into the field, a drop down list populates with prior entries. Each subsequent person has access to a growing drop down list of options to select or can enter their own. This basically let’s the person doing the data entry select qualitatively the category that best fits their answer.

The statisticians have commented positively about how this has impacted their work. “It used to be that I would spend hours reading through a list of text entries and try to make sense of them by categorizing them into groups for analysis. Using VISION’s Autocomplete function has greatly reduced the time I spend on that. People select existing options categorizing their entries. All I have to do is look at the ones that weren’t selected frequently and decide whether to have an ‘other category’ or code them into existing categories. It is so much easier.”

As a result of using the new Autocomplete feature, data is more streamlined, data analysis time is shorter and more efficient, there are less errors and less is left up to the statistician’s subjective analysis. The people doing the data entry report they appreciate being able to select from existing options instead of entering the same text over and over again.

In the end, the data is cleaner and can be analyzed quicker. This improves confidence in the results of the study and reduces time to market for new drugs.

About Prelude Dynamics & VISION™

Prelude Dynamics is a global provider of customized web-based software systems for data collection, analysis and management of clinical trials, studies and registries. We streamline eClinical operations through our unique and innovative VISION™ software system, which allows us to rapidly configure data collection solutions for pharmaceutical, CRO, medical device, animal health, and university organizations.

Handling a Surgical Study with Repeated Observation Times

 

Challenge

 

While collecting repeated observations of animals using a standardize scale is not in itself a challenge, allowing access to only one scale at a time during the appropriate time frame to the correct individual is a challenge. For this particular study, up to three individuals could conduct the observations around the clock at specific intervals over a minimum of a 72-hour time-frame and were restricted to conducting observations at specific times. The scale needed on-demand functionality as well in case additional observation times were necessary. The clock for the observation period started running off the surgical start time, which also had to be incorporated into the solution. Additionally, if the score was above a certain threshold, the researcher had to be alerted that an alternate procedure (medication or treatment protocol) had to be followed and/or the animal had to be removed from the study.

 

Proposal

 

Prelude Dynamics proposed utilization of VISION™, an electronic data collection system, to allow researchers to capture information in real-time, provide the standardized scales in a just-in-time manner, follow the natural work flow of the study process, and ease the data collection process. In order to restrict the scales to the appropriate individuals, form level permissions would be assigned to specific roles. VISION™ would be programmed to use custom calculations to automatically start the clock from the surgical start time entered and provide the correct dosing amount of any medications administered.

 

Background

 

It is a common study objective to understand how animals respond to a certain surgical procedure, anesthesia, or post-surgery medications. Researchers examine an animal’s response at specific time intervals over a period of time to reach this understanding. When this time period extends over multiple days, it is necessary to utilize multiple observers to complete standardized scales.

 

When the data is collected on paper, it is difficult to blind current observers from information completed on forms by a previous observer. Access to previously entered information can bias the current observer, regardless of whether he or she collected the prior observation or it was collected by someone else. Reducing bias is important to obtaining reliable information.

 

Solution

 

Prelude staff trained the study staff to utilize VISION™ to show and provide access to only one scale at a time, restricting access to the individual with the appropriate permission at the designated time. VISION™ was programmed to start running the time clock off the entered surgical start time. Permissions were assigned to roles that corresponded with access to particular scale forms as indicated in the study protocol. Some scales could only be filled out by a certain role while others had multiple roles that could fill them out. Once a scale was filled in, reviewed and approval given, that scale became inaccessible or hidden and the next scale became accessible. Based on the initial entered surgical start time, each scale displayed the expected date and time that scale needed to be filled in.

 

As a standard, VISION collects full audit trail information on each data point entered into the system, including who entered the information and a date and time stamp. It also allows data entry fields to be augmented with comments or notes, which is an important feature if additional information needs to be added that might otherwise be limited to a selection of choices.

 

VISION™ also contains pre-existing summaries which assist in monitoring the data being collected at the patient level, at each site or for the study as a whole. Using VISION™’s search and filter capacity allows these summaries to display data of only the cases fitting the search criteria. Additionally, VISION™ provides users with certain permissions the ability to create ad-hoc reports which they can use for monitoring additional criteria or in order to answer important research questions. The data within the system is available for export 24/7/365 with the appropriate user permission. Export features the ability to export in multiple formats including SAS-ready format.

 

Results & Benefits

 

Real-time data is being entered into VISION™, and standardized scales are being filled in and signed off on, the completed form is no longer accessible to the individuals completing forms.  Using VISION™ eliminates observer bias since current observers cannot view or compare findings entered by a prior observer. There are certain roles designated by the Sponsor to maintain read-only access to view all the scales at any time.

 

Scale scores are immediately calculated and feedback automatically provided if scores indicate necessity for removal from the study or other treatment. This identifies potential problems quickly and reduces the risk to the participating animal.

 

VISION™ also identifies these animals quickly so they can be taken out of the study and prevent further costs associated with tracking an ineligible subject. Additionally, when an animal is removed from the study, the VISION™ Randomization module, which is behind the scenes, will reassign the withdrawn animal’s treatment to a new case to ensure that sufficient cases are enrolled for each treatment group.

 

The need for data collection on paper CRF has been eliminated. Researchers spend less time entering data after the fact because they are able to enter information directly into the system in real-time. There is also a cost savings and error reduction by eliminating data entry at a later date.

 

Data is accessible in real time for query, review, analysis and export. Users report the streamlining of the process has helped reduce errors and improved efficiency

Prelude Enhances Automated IDEXX and Marshfield Lab Data Representations to Include Both Tabular and Non-Tabular Formats

AUSTIN, TX – April 14, 2016 – Prelude enhances automated IDEXX and Marshfield lab data representations to include both tabular and non-tabular formats.

Data managers and Biostatistitians now have a choice when exporting lab data in studies that use IDEXX and Marshfield labs as their lab of choice. The data can be exported in a tabular or non-tabular format, whichever suits their need for analysis.

This is an important enhancement as the automated computer-to-computer lab interface that allows both IDEXX and Marshfield to send lab data directly to VISION™ provides the data in a non-tabular format. Previously only the non-tabular export format was available.

This new enhancement translates the non-tabular format to a tabular format that often is preferred by data managers for use in their analysis. While data managers and biostatisticians can do this conversion themselves within their statistical software, it is a time consuming task that no longer needs to occur with the use of this enhanced automated lab interface.

Prelude Dynamics is an industry leader at providing innovative software-as-a-service technology solutions for clinical trials across the globe. The company’s flagship product, VISION, offers secure and configurable data capture and information awareness tools for the advancement of human and animal health, compliant with FDA/CVM and EMEA regulations. VISION™ facilitates real-time data collection, querying, cleaning, reporting, and analysis all based on the needs of the researchers. The company was established in 2003 with a goal to speed the study process, improve data quality, and facilitate easy data entry through automation.

Contact

Jim Pedzinski – VP of Business Development
jpedzinski[at]preludedynamics.com – 512-476-5100 ext. 210

VISION™ Feedback Tool Has Been Enhanced to Support Client Approval and Testing Documentation

AUSTIN, TX – March 10, 2016 – Prelude announced today that it has enhanced its feedback tool. Clients use VISION’s feedback tool during the study build process to communicate changes and needs to their Prelude Project Manager. Feedback is integrated into the study and allows users to leave feedback directly on the pertinent field.

When feedback is left, as with all data entered into VISION, there is a full audit trail of who left the feedback and when. The feedback tool tracks the communication between the Sponsor and Prelude related to study design needs.

As expressed by one of our clients, “We appreciate the iterative development study process and relationship we have with Prelude. The feedback tool allows us to quickly note any changes. It keeps track of the changes we have requested, and allows us to have a team of people testing and interacting with the study during the development phase. As the PM in charge of this process, this enhancement allows me to approve the suggested changes before Prelude implements them. And the audit trail allows me to see which of our staff entered the change so I can follow up with them if I have questions prior to approving the change.”

The enhancements include addition of a change approval column and a reviewed column. The change approval column is intended to be used by the project lead to indicate whether the suggested change should or should not be made. This is important because many clients engage teams to collaborate in and test the study build. Each of these team members comes from a different perspective and might make different suggestions based on their experience. Some of the changes suggested might not be needed or might be in conflict with other changes, so this allows the project lead to review and designate each change as desire prior to Prelude’s implementation.

The reviewed column allows clients to better track and review the requested changes. Once the change is made, the Client needs to ensure the change is made as desired. Many clients were tracking this offline on excel sheets because it was an important quality assurance process. Now, they can quickly indicate their review directly in VISION™ which gives them a full audit trail of their approval.

Additionally, the testing documentation provided to clients as a suggested starting point for testing their study has been enhanced. It is now available as a pop-up window rather than a download. This makes it easy to have the testing document beside the study to ensure thorough testing.

Prelude Dynamics is an industry leader at providing innovative software-as-a-service technology solutions for clinical trials across the globe. The company’s flagship product, VISION, offers secure and configurable data capture and information awareness tools for the advancement of human and animal health, compliant with FDA/CVM and EMEA regulations. VISION™ facilitates real-time data collection, querying, cleaning, reporting, and analysis all based on the needs of the researchers. The company was established in 2003 with a goal to speed the study process, improve data quality, and facilitate easy data entry through automation.

Contact

Jim Pedzinski – VP of Business Development
jpedzinski[at]preludedynamics.com – 512-476-5100 ext. 210

Improving Inter-Rater Reliability

 

Challenge

 

When multiple raters will be used to assess the condition of a subject, it is important to improve inter-rater reliability, particularly if the raters are transglobal. The complexity of language barriers, nationality custom bias, and global locations requires that inter-rater reliability be monitored during the data collection period of the study. Identifying these deviations and retraining staff as quickly as possible would improve the quality and consistency of data collection.

 

In addition to addressing the challenges of raters being located globally, the data review monitors may also be located globally, and the study project manager and supervising monitor may also be in another global location.  The client was seeking to identify an Electronic Data Capture (EDC) system that could not only manage the data collection, but could monitor multiple raters for inter-rater reliability. One way to do this would be to compare the score assigned to the observation to photos of the observed area, which would require the ability to upload and download photos, maintain metadata which ties the photos to a specific subject and visit date, allow feedback/queries and their responses, and provide a full audit trail of all data in the system. Ideally, the system would also notify the monitors that rating and photo data is entered and is ready for follow up if any discrepancies or potential inter-rater reliability issues are identified. Similarly, the system would also notify the rater when a feedback/query was entered and needed the rater’s response. Finally, the monitor would be notified when feedback/queries were resolved.

 

Proposal

 

Prelude Dynamics proposed the client use VISION™, an electronic data capture system, which already provides the ability to customize the electronic Case Report Forms to collect the data necessary for the study. VISION™ could be customized to collect basic subject information, physical exams, patient medical history, visit history, study scales, and capture photos of the observation areas.  VISION™’s notification system could be programmed to notify monitors of observation and photo entries as well as resolved queries and notify the rater of queries needing to be resolved.

 

Additionally, VISION™’s Randomization Module could provide just-in-time randomization globally across the study to ensure it would be difficult to predict the subject’s treatment group or it could randomize at the site level depending on the study design. The randomization module could also allow the stratification of subjects based on specific criteria, depending on the study’s needs.

 

The Inventory Management Module could be used to track medications and supplies throughout the study. The inventory module could even automatically place orders to ensure the minimum level of supplies and medications would be on hand. Ad-hoc reports and summaries would assist the monitors in keeping track of the study progress as well as monitor inter-rater reliability.

 

Background

 

When multiple clinical sites are used to collect data, it is necessary to train multiple raters, which requires taking the extra step to ensure inter-rater reliability. Assuming that the scale is sensitive enough and appropriate to use in a multiple-rater study, then any identified inter-rater reliability issues would require retraining of the rater.

 

It is often difficult to identify inter-rater reliability issues, particularly when a rater needs to be retrained. When the rater and the monitor are located remotely from each other, prompt monitoring is difficult because the monitor may only be able to review the data when they make a site visit, or the monitor may have access to the data, but not the related photos that are being used to monitor the observation. The necessity to retrain staff can incur costs to the study, and there are often multiple deviations of data as scales are rescored or removed from the study altogether, which is also costly.

 

Solution

 

Prelude Dynamics trained the staff to utilize VISION™ to capture all study data, upload photos, and take advantage of the system’s custom notification capability .  Raters use tablets to record rating observations directly into the system. As the raters work through various data collection and observation screens, diagrams or illustrations will remind them of areas to be observed and the details of the rating scale, and a rating section will be provided. When scales are completed, photos can be uploaded into VISION™ to represent the area observed. In order to maintain study and data integrity, once submitted, the rater cannot change the scales. In fact, the scales will no longer be accessible via a tablet, and will only be visible to the rater via a read-only summary table which has captured the scale input. The Monitor can view the summary table and uploaded photos, and can compare them to the rater’s input and query the information as necessary. The solution also included:

 

  • Standard notifications to notify the Monitor and Project Manager of the following events:
  • New patient enrollment (i.e., patient successfully randomized)
  • Patient terminated from study
  • Changes to form state from in-work to in-review, in-review to reviewed, reviewed to finalized
  • Query resolved
  • Serious adverse events occurred
  • Observation scale and/or photos uploaded
  • Custom notifications would be created to notify of the following events
  • Query entered by Monitor (notify the site personnel)

 

VISION™’s Inventory Management Module will be used to keep track of both medication and supplies for this study. The module will also be set up to automatically order medication and supplies when they meet the minimum threshold to ensure the study progress is not compromised.

 

VISION™ additionally provides mid-study monitoring capacity through pre-set summaries, search/filter, and ad-hoc reporting. One of the standardized summary reports contains a listing of all queries both resolved and open. This will assist the monitor in keeping track of the progress of the study. For additional mid-study monitoring requiring a statistical package, export is available 24/7/365 for those roles with the appropriate permissions.

 

Results & Benefits

 

As a result of using VISION™ to capture study data, the Monitors are able to better track the study progress and better able to ensure inter-rater reliability. The monitors receive emails notifying them that a new scale and photos have been uploaded. They are able to manage their time better and provide prompt feedback. Designated site personnel receive an automatic email notifying them if a Monitor enters a query that needs resolution. Queries are promptly resolved when information is fresh in the researcher’s mind. As a result, the quality and reliability of the study data is improved.

 

The overall or primary Monitor is able to communicate directly and without delay with globally located site/country Monitors about any concerns over the scale ratings. The primary monitor has access to all scales from all sites, including photos and scale results, and can communicate to resolve any concerns or discrepancies identified. This helps ensure overall inter-rater reliability.

 

The Monitors and site report that data and query notifications through the system saved time, ensured quick responses, and kept them connected to the data and personnel through the study. This is especially important because all to the participating sites are active busy clinics that have their own patients and not just study participants.

 

Additionally, the site found it very valuable to have VISION™ keep track of the inventory, citing the efficiency of a system which automatically updates as patients are provided with medication and supplies, rather than keeping separate track of inventory. The automatic reordering feature reduced the stress associated with ensuring they always had enough supplies on hand to continue study progress. The Project Manager also found this feature reduced stress and logistics that otherwise would have posed a headache to handle. Imagine ordering supplies for different sites in different countries and keeping track of that! It would have been a full-time job just to do that.

 

The Project Manager and primary monitor tracked the study using the summaries and ad-hoc reports, staying up-to-date with the enrollment and progress at each site. They were also able to review the mix of subjects to ensure there were going to be sufficient subjects of each age and gender, and specifically noted how easy it was to copy and paste a report from VISION into a spreadsheet so they could provide monthly updates to the Sponsor.

 

Overall, everyone participating in the study found that VISION™ made the study run more efficiently and reduced their stress over participation in the study. Edit checks ensured the data was collected appropriately, and immediate feedback for missing or out-of-range data could be immediately collected  or verified as correct. They also found that it greatly improved inter-rater reliability which gave them more confidence in the data they collected throughout the study.

 

About Prelude Dynamics & VISION™

 

Prelude Dynamics is a global provider of customized web-based software systems for data collection, analysis and management of clinical trials, studies and registries. We streamline eClinical operations through our unique and innovative VISION™ software system, which allows us to rapidly configure data collection solutions for pharmaceutical, CRO, medical device, animal health, and university organizations.