When implementing AI-driven solutions in IT support, accuracy and precision are paramount. This is especially true when it comes to ticket classification, where ensuring proper categorization directly impacts response times and resolution accuracy. CrushBank’s SmartClassifier is designed to take the burden of manual classification off technicians’ shoulders, streamlining operations and allowing IT support teams to focus on solving issues rather than sorting them. However, achieving high classification accuracy requires an iterative and ongoing refinement process. This article will dive into how CrushBank continually refines its SmartClassifier to achieve optimal results for Managed Service Providers (MSPs).
The Starting Point: Enabling SmartClassifier
When SmartClassifier is first enabled for all “New” incoming tickets, it operates with an initial accuracy rate of approximately 65% or better. This baseline can vary depending on how consistently and accurately tickets were classified in the past by the client. Despite this variation, CrushBank almost always sees immediate improvements, as its classifier algorithm benefits from the existing data.
From this initial deployment, the classification accuracy increases rapidly through a structured, four-week refinement process. This involves collaboration between the client’s IT team and CrushBank’s AI specialists to refine and fine-tune the classifier’s model.
The Four-Week Refinement Process
The key to SmartClassifier’s success lies in its four-week iterative process, during which both CrushBank and the client take an active role.
1. Week 1:
– CrushBank’s Role: SmartClassifier is activated for all new tickets, and the system begins classifying them based on the available historical data.
– Client’s Role: The client’s IT technicians and managers begin paying close attention to classifications. As they work through tickets, they confirm or update incorrect classifications. This feedback directly informs the re-training of the model, enhancing its performance.
2. Week 2:
– CrushBank’s Role: Based on the classification data generated during Week 1, CrushBank creates an updated classification model. By incorporating client feedback and classification changes, the system’s consistency and accuracy continue to improve.
– Client’s Role: At this stage, the client performs a review of the updated classifier. The Help Desk Manager (or a similar role) dedicates approximately one hour per week to reviewing closed tickets, ensuring classifications are accurate, and discussing any adjustments with CrushBank.
3. Week 3:
– CrushBank’s Role: Weekly updates continue as needed, with CrushBank monitoring performance and generating further model refinements. Any new classifications or adjustments made by the client are incorporated into subsequent models.
– Client’s Role: The technicians keep updating any misclassified tickets as part of their normal workflow. The Help Desk Manager reviews the performance of the updated model and provides further feedback to CrushBank.
4. Week 4:
– CrushBank’s Role: The final stages of the initial refinement process occur. After Week 4, SmartClassifier is now enabled not only for new tickets but also for ticket closures, enhancing classification accuracy to around 90%-95%.
– Client’s Role: Technicians no longer need to manually classify tickets. However, if new ticket types, subtypes, or items are created, the client notifies CrushBank after classifying at least ten tickets with the new options. This ensures the classifier can adapt to evolving ticket structures.
Beyond Week 4: Continuous Improvement
Even after the four-week refinement period, the classifier continues to be fine-tuned. When clients introduce new types, subtypes, or items, CrushBank updates the model to incorporate these changes. This ongoing communication ensures that SmartClassifier stays up-to-date, consistently delivering high accuracy as ticket categories evolve over time.
A Win for IT Support Teams
By the end of this refinement process, SmartClassifier is operating at high rates of accuracy, eliminating the time technicians or operations people spend manually classifying tickets. This frees up their focus for more critical tasks, such as resolving issues faster and improving overall service quality. Additionally, as the system continues to learn from new ticket types and feedback, classification accuracy remains high, even as the MSP’s operations and ticket types evolve.
The Bottom Line: AI + Human Input = Superior Accuracy
While SmartClassifier starts strong out of the gate, it is the combined effort of AI and human feedback that propels it toward near-perfect accuracy. CrushBank’s structured refinement process ensures that the classifier is always learning, adjusting, and improving, making it a powerful tool for MSPs looking to streamline their ticket classification processes.
By working together with clients, CrushBank ensures that SmartClassifier is tailored to the specific needs of each MSP, resulting in better data, faster ticket resolution, and more efficient IT support operations.
By integrating this ongoing refinement process into your MSP’s operations, you can ensure that your classification accuracy stays at its peak while your technicians are freed to focus on delivering top-notch support.