The Trial Master File (TMF) allows an effective way to collect and manage study specific documents during a clinical trial. The TMF serves as a complete and accurate record of the trial's conduct, including the protocol, informed consent forms, study reports, and other important documents. With the increasing volume of documents generated during clinical trials, managing them manually is inefficient, time consuming, and often overwhelming.
To help alleviate this problem, Artificial Intelligence (AI) with automated classification schemas can be configured to auto-classify and manage the documents generated during clinical trials, driving efficiency, and ensuring greater consistency and quality.
This blog post shows how ProPharma works with study sponsors to drive TMF efficiency through automation. ProPharma's R&D Technology Solutions team specializes in digital transformation, including the implementation and configuration of a wide array of clinical and research systems. This use-case explores optimizing Veeva Vault eTMF to enable an AI-powered solution, the TMF Bot, to automate the classification of documents.
AI and automated classification schemas can revolutionize document management during clinical trials.
What is Veeva Vault's TMF Bot?
The TMF Bot is an AI-driven solution within Veeva Vault that employs natural language processing (NLP) and machine learning (ML) algorithms to automate the categorization of documents within an electronic Trial Master File (eTMF). By examining the content of each document uploaded through the Document Inbox, the TMF Bot discerns its classification and automatically assigns the appropriate category, leveraging insights gained from trained models. Benefits of Implementing Veeva's TMF Bot.
Improved Accuracy and Consistency
One of the main advantages of leveraging Veeva Vault's TMF Bot for eTMF classifications is improved accuracy and consistency. The manual classification of documents tends to be time-consuming and frequently leads to misclassifications. However, by employing the TMF Bot, documents can be accurately and consistently classified, effectively streamlining eTMF management and enhancing the quality of the eTMF.
Increased Efficiency
Leveraging Veeva Vault's TMF Bot for eTMF classifications also increases efficiency by enabling faster document processing. By utilizing the TMF Bot, documents are automatically classified, minimizing the requirement for manual intervention. This eliminates the need for valuable resources to be spent on classification tasks, enabling study teams to concentrate on more critical responsibilities. Trial performance improves and products can get to market faster as a result.
Holistic Approach to TMF Bot Implementation
ProPharma's R&D Technology Solutions team follows a comprehensive approach to assist study sponsors in implementing and launching their TMF Bot. Throughout the entire process, ProPharma offers customized support to ensure the sponsor's trained model implementation delivers maximum value.
TMF Health Check
Each study sponsor has distinct requirements for document management. Therefore, it is crucial to conduct a comprehensive quality assessment of the documents stored in your Vault and evaluate the overall condition of TMF documents before deploying the TMF Bot. This assessment gathers essential baseline data that enables the customization of the trained model and determines the most suitable approach for implementing the TMF Bot, tailored to the specific needs of the sponsor.
Create a Trained Model
ProPharma collaborates with study sponsors to develop a trained model, which is a comprehensive record that documents the specifics of an ML model either already created or planned for a sponsor's Vault. When constructing a trained model, there are several crucial parameters that need to be established, including the Prediction Confidence Threshold and Minimum Documents per Document Type. As different companies have diverse risk tolerance levels, these parameters can be tailored to match each company's corporate objectives, allowing for customization of the trained model to align with their specific needs.
Select Documents for Training
Accurate document classifications are extremely important for the success of a clinical trial. After analyzing the overall health and composition of a study sponsor's documents, ProPharma's specialists offer a recommendation regarding the optimal approach for selecting training documents for the TMF Bot. This recommendation may involve determining the Training Window Start Date or utilizing an attached Computer System Validation (CSV) of Document IDs.
By employing these methods, the aim is to maximize the accuracy of the TMF Bot's automated document classification when it is initiated.
- When study sponsors intend to utilize the TMF Bot and deploy an optimal trained model, it is important to consider the following key factors: Recommend using a minimum of 1,000-3,000 accurately classified steady-state documents in the Vault (Approved/Final) to increase the prediction quality (maximum of 200,000 documents)
- A minimum of 10 documents within each document type (e.g., Trial Management, Central Trial Documents, Site Management, Investigational Product and Trial Supplies, Data Management, etc.) if loading between 1,000- 10,000 documents (minimum increases if loading 10,000+ documents)
- Exclude non-English, video, audio, ZIP, statistical, and database files
- Exclude Document Types mapped to TMF Document, Final CRF, and Sites Evaluated but not selected
The more accurately classified document classifications are, the more accurately trained the TMF Bot will be when deployed.
Evaluate and Deploy
After training the trained model, ProPharma collaborates with study sponsors to review and analyze the Training Summary Results, Model Performance Metrics, Extraction Coverage Percentage, and Individual Predictions. Our trained experts work closely with study sponsors to ensure that all summary results meet expectations before assisting in the deployment of the trained model.
Following deployment, the TMF Bot initiates the automated classification of documents within the sponsor's Vault, facilitating the automation of crucial business processes for study teams.
Post TMF Bot Deployment Surveillance
ProPharma's R&D Technology Solutions team understands that deploying the TMF Bot is only the beginning of the journey towards successful implementation. Customized support is available to guarantee study sponsor satisfaction by determining the most suitable frequency for spot-checking documents that have been auto-classified by the TMF Bot, closely monitoring prediction data, and deploying additional trained models as required. Veeva Vault TMF Bot Implementation and Optimization
Once sponsors have launched the TMF Bot, ProPharma helps unlock the full potential of their Vault's TMF Environment.
Prediction Data Review
Once the trained model is deployed, ProPharma realizes the importance of ensuring TMF Bot adequacy for the content uploaded to the Document Inbox. A few weeks after deployment, ProPharma collaborates with study sponsors to review the Predictions Report to confirm the trained model is performing as expected.
ProPharma provides ongoing monthly or quarterly quality/verification checks of sponsor's Predictions Report, depending on the agreed-upon cadence.
Conducting these reviews ensures the trained model continues to meet the requirements of the uploaded content, while also identifying any content that requires reclassification. This iterative process employed by ProPharma empowers study sponsors to optimize their utilization of the TMF Bot, thereby ensuring its continuous success.
General Release Gap Analysis
Staying up to date with the latest innovations and functionalities of the Veeva Vault Platform is of utmost importance to ProPharma. For study sponsors who utilize the TMF Bot, in addition to our impact analysis service for every General Release, we offer a comprehensive gap analysis and risk assessment prior to Veeva Vault software updates. This ensures that sponsors can fully leverage any new enhancements related to the TMF Bot in upcoming releases. Through this ongoing evaluation and update process, study sponsors can have confidence that their eTMF is optimized for utmost efficiency and accuracy, enabling them to make the most of the TMF Bot's capabilities.
Want to Implement AI for Your Trial Master File?
If you are looking for a trusted partner to implement Veeva's TMF Bot or other Veeva Vault software and enhance the quality and consistency of your TMF documentation, ProPharma's R&D Technology Solutions team is here to help. Our team of experts has the knowledge and experience to seamlessly integrate Veeva's TMF Bot with your existing systems and provide ongoing support to ensure your success.
Contact us today to learn more about how we can help you streamline your TMF documentation process and achieve your regulatory compliance goals.
