Seven common AI training challenges and how to address them

As the applications of AI proliferate, it becomes increasingly important to acknowledge and address the many challenges faced by those on the front lines of AI training and development. These go well beyond 'mere' practical constraints (though there are certainly many of those). Most of the challenges inherent in AI development reflect societal concerns, and our fast-growing recognition that we need principles to guide the responsible development of AI. The seven common AI training challenges covered here show just how wide-ranging these concerns are and how important it is to take responsible AI development seriously.

If training data is the foundation of AI development, then data acquisition is one of the most fundamental AI training challenges (exacerbating many of the other challenges). AI training requires vast amounts of good-quality, relevant data to deliver an AI application that meets users' increasingly high expectations. The first challenge – even before considering data quality – is how to source enough of it. Especially for niche applications, the required volume of data may not even exist. If it does, it may be difficult to acquire due to privacy or legal restrictions (see also challenge no. 2 regarding privacy).

What to do? 

Any or all of these approaches could help you address this particular AI training challenge: 

• Public datasets. Look for publicly available or open datasets, such as those provided by government or research institutions. Make sure that you use only reputable data sources, in line with the responsible AI principles of safety and reliability. 
• Data collaboration. Partner with other companies or research institutions to share data. This increases the pool of data available for AI training, but it's vital that all parties are serious about doing so responsibly and legally. 
• Data augmentation. Modify existing data to increase the size of the training dataset. For instance, an image can be flipped, rotated, zoomed or cropped to create new images. 
• Synthetic data. Manufacture entirely new data using an algorithm or simulation that can produce output that closely mirrors real-world data. 
• Outsourcing. Work with a professional and responsible provider of AI training data services, such as TrainAI by RWS, to fulfil your unique AI data requirements.

AI training often require the use of datasets that include personally identifiable information (PII) such as names, phone numbers or addresses, or sensitive information such as health data, financial records or confidential business information. If you have no choice but to use such data, you must do so without compromising the privacy of individuals or organizations. This AI training challenge is both an ethical and a legal one, since beyond the principles of responsible AI there are also data protection laws to comply with. 

What to do? 

Any or all of the following approaches will help you address this particular AI training challenge: 

• Data minimization. Reduce the chance of sensitive information being unnecessarily exposed by collecting the minimum amount of data required for AI training. 
• Data encryption. Encrypt data in transit and at rest to protect it from unauthorized access. 
• Data anonymization. Strip the data of PII to ensure that individuals cannot be identified. 
• Differential privacy. Inject noise into the data in such a way as to mask individual information without significantly impacting the overall accuracy of the model. 
• Federated learning. Instead of transferring data to a central model for training, send the model to the data to learn, so that the data never leaves its original location. 
• Privacy policies. Establish clear policies that explain how data will be collected, used and protected to maintain the privacy, trust and consent of the individuals whose data is being used. Make sure that your policies and actions comply with all relevant data protection regulations, such as the GDPR, to ensure that data privacy requirements are respected throughout the AI training process.

The well-known concept of 'garbage in, garbage out' really does sum up the relationship between the quality of training data and the performance of your AI model. So how do you ensure that you're doing the opposite, namely 'quality in, quality out'? This is one of the toughest AI training challenges, not only because of the volume of data involved, but because of the many aspects of AI training data quality. So much so, that we've covered the topic of data quality separately. 

What to do? 

Read our blog on data quality for the detail on these approaches: 

• Data governance for quality oversight 
• Data verification and cleansing for removal of errors and inconsistencies 
• Feature selection of only relevant data features required for AI training 
• Data audits and continuous improvement for ongoing quality control 
• External help using third-party datasets or an AI data service such as TrainAI by RWS

As our separate blog on data quality makes clear, one of the most crucial characteristics of data quality for AI training is that it be bias-free. I've called it out here because it's such an important and specific AI training challenge. If the training data includes biases, consciously or unconsciously, the AI model is likely to replicate those biases in its predictions. For example, a facial recognition system trained on a dataset that predominantly features people of one ethnicity may struggle to accurately recognise individuals of other ethnicities. 

What to do? 

It takes a comprehensive toolkit to address this AI training challenge, including: 

• Ethical guidelines. Establish and implement clear guidelines in line with principles of responsible AI to ensure that the AI model does not perpetuate harmful biases. 
• Diverse and representative data. Actively seek data which is fully representative of all the groups that will use or be impacted by the AI model, covering a broad range of relevant scenarios, outcomes and characteristics to help the AI model make unbiased decisions. 
• Transparent algorithms. Use transparent or interpretable methods to develop the AI model, so that it’s easier to scrutinize for potential bias. 
• Continuous monitoring for bias. Conduct regular audits to identify any potential bias in the AI model, for subsequent mitigation. You can use fairness performance metrics for a quantitative measures of a model's bias, for example demographic parity (measuring the probability of the AI model treating all demographic groups equally, without favouring one over another) or equalized odds (demanding that the model's true positive and true negative rates are the same for all demographic groups). 
• Bias mitigation techniques. During data preprocessing and model training, address any biases discovered in the model with techniques such as data re-sampling or other modifications of training data (pre-processing mitigation), modification of the learning algorithm itself (in-processing mitigation), or modification of the model's output (post-processing mitigation). 
• Human in the loop (HITL). Incorporate human oversight, cross-checking and quality assessments, and reinforcement learning from human feedback (RLHF) to detect and correct biased AI output.

Too often, AI applications have a black-box problem, which makes it impossible to understand how the AI model processes data or generates its output, or to explain the decisions it makes. It's a real problem for combatting bias, and perhaps the most important of the AI training challenges to solve if we want people to trust in AI. 

What to do? 

AI transparency is so important for the development of responsible AI, that we need to address it using a range of different approaches: 

• Training data transparency. Always maintain detailed information about AI training data, not just its characteristics but its source and preparation, including its treatment during preprocessing. 
• Documentation. Keep detailed records of every step in the model's design, training and deployment, to provide valuable context for, and understanding of, the model's decisions. 
• Feature importance analysis. Gain insight into the factors driving the model's predictions by using feature importance scoring techniques to calculate and analyze the contribution of each feature of the data to the model's output. 
• Interpretable models. To ensure that the inner workings or predictions of the model can be understood and articulated, employ methods that clearly reveal how its inputs are linked to its outputs, such as decision trees or linear regression. 
• Explainable AI (XAI). Design the model to produce explanations of its outputs. Use techniques such as LIME (Local Interpretable Model-Agnostic Explanations) or SHAP (Shapley Additive Explanations) to generate explanations for the model’s predictions or decisions. 
• Human in the loop (HITL). Involve humans with the expertise to interpret and validate the decisions made by the AI model.

The more AI is embedded into our lives, the more important it becomes for the applications to be able to adapt to changing environments and learn from new data in real time. Otherwise they will inevitably 'fall behind', producing predictions and decisions that increasingly become less relevant. 

What to do? 

Solving this AI training challenge requires advancements in online, incremental learning techniques, among other continuous learning models. Techniques to consider include: 

• Model retraining. Regularly retrain models with recent data to ensure their relevance and accuracy. Unlike incremental learning (below), this involves ‘starting over’, not building on previous learning. 
• Data streaming. Send a continuous influx of new data to the model to facilitate real-time training. 
• Incremental learning. Use techniques that allow the AI model to keep building on its learning with incremental additions of data, while retaining its previous learning (i.e., without letting the new data drown out what has already been learned). 
• Adaptive learning systems. Develop systems that can adjust their learning strategies based on the changing data environment. 
• Human oversight. Incorporate human supervision into the learning process, to monitor the AI's adaptation and intervene when necessary.

Training modern AI models takes a huge amount of time and computing power – and therefore energy. This has obvious implications for the sustainability objectives of individual organizations and society in general. 

What to do? 

Given the size and complexity of today’s AI models, we need to find ways to train them more quickly, with less data, using less energy, without compromising their performance. Examples of ways to do so include: 

• Efficient algorithms. Choose the most efficient of the potential learning algorithms for the model. Some methods use less data and computational resources than others for comparable performance. 
• Transfer learning. Leverage a model trained to perform a similar task, allowing you to adapt it to the task you need much more efficiently than starting from scratch. This reduces both training time and computational resources used. 
• Pruning. Remove unnecessary components from the model, such as neurons in a neural network or branches in a decision tree, to reduce model size and computational requirements without affecting model accuracy. 
• Quantization. Reduce the numerical precision of the model by representing numbers with fewer bits (lower precision) without significantly compromising the model's performance. This slims the model down, reducing memory and computational power requirements. 
• Energy-efficient hardware. Use hardware for AI training that is known for its efficiency, such as setups including application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). 
• Eco-friendly data centres. Opt for environmentally friendly data centres for AI training and processing, such as those powered by renewable energy. 
• Carbon emission tracking. Raise awareness and promote more sustainable practices by using tools that measure and track the carbon footprint of AI training processes.

None of these AI training challenges is easy to overcome. But as with all challenges, we should regard each as an opportunity to drive innovation – innovation critical for the development of AI that is ethical, effective and beneficial for all. 

Ready to get started? Contact RWS’s TrainAI team for AI training data services that can help you address these challenges.