Bright Box Demonstrative Realization: Knowledge Pit

Knowledge Pit is the only platform in Poland for organizing contests in the field of data science. It currently has around 2000 registered participants from all over the world and one of the most popular competitions has received over 150 entries.

The Knowledge Pit platform provides its users with a unique possibility to analyze in detail results and algorithms in order to find out about their behavior, strengths and shortfalls. Integration with the BrightBox tool set provided by QED Software makes it possible to take advantage of the techniques belonging to the domain of eXplainable Artificial Intelligence (XAI).

With XAI tools from the BrightBox arsenal, once the challenge concludes and the complete data is released, the participant can perform:

• diagnosis of errors made by their constructed AI/ML models together with the identification of the most probable error causes;
• verification of validity and optimality of the chosen approach;
• identification and analysis of the most probable error causes;
• identification of the most problematic data samples;
• detection of overfitting/underfitting characteristics of AI/ML models constructed during the challenge;
• discovery of the constructed models’ behavior related to problematic data, outliers, missing information and so on;
• visualization of various aspects of the operation of their AI/ML models on complete data sets OR a complete data set.

For the challenge organizers and challenge providers and owners the integration of BrightBox eXplainable Artificial Intelligence (XAI) tool set presents numerous opportunities. With XAI methods applied to AI/ML models submitted as solutions to the challenge it is possible to:

• provide collective explanations for decisions yielded by constructed (trained) AI/ML models that were submitted by participants;
• gain extended understanding of data and processes, e.g., identify errors and outliers in data sets;
• perform ‘What-if’ analysis and detection of optimization possibilities for processes controlled by AI/ML models.
• perform more extensive and multi-dimensional diagnosis of errors made by multiple AI/ML models together with identification of the most probable error causes;
• design more interpretable learning/training methods for AI/ML models, possibly as a combination/hybrid of solutions submitted to the challenge.