1. Unethical Data Sourcing Concerns have defined the Initial Stages of Generative AI Training

Kenya’s emergence as a hub for artificial intelligence (AI) innovation offers immense potential, especially for addressing societal challenges. For instance, AI-powered Assistive Technologies (ATs) hold transformative possibilities for disability inclusion.¹ AI in Kenya has also positively impacted education, agriculture, health and business sectors.² Unfortunately, most of the AI tools used in these sectors in Kenya originate from large foreign technology companies and organisations with the financial resources to develop them.³ Kenyan developers often rely on these foundational tools and datasets to build local solutions due to inadequate quality datasets fully reflective of local populations and concerns.⁴ AI policy researchers have highlighted the significant problem of a lack of high-quality datasets that accurately represent local populations. This gap exacerbates biases in AI systems, as flawed training data—often sourced from Western contexts—fails to account for diverse demographics. A striking example of this issue is Joy Buolamwini’s 2021 documentary Coded Bias, where she and other AI and data stakeholders uncovered that AI systems struggled to detect darker faces and inaccurately categorised women.⁵ This revelation underscored how algorithms reflect real-world biases shaped by historical data and the limited diversity within development teams. Consequently, local datasets are recognised as a crucial opportunity for African developers to create AI solutions tailored to the continent’s unique challenges.

While stakeholders have embarked on the well-intentioned mission of collecting and preparing data for training AI systems, this process has faced challenges that have long plagued relations between Africa and the rest of the developed world.⁶ A key issue is the tendency of foreign companies—and even local contractors—to extract data from local populations at little or no cost, only for the resulting technology systems to be sold back to these same communities at a price. This “digital colonisation” presents itself in Africa in various forms, such as through cheap digital labour, data extraction, and Africans being relegated to the roles of beta testers of global platforms.⁷ The challenge of social media data is particularly relevant; since the rise of platforms like Facebook, Twitter, Instagram, and TikTok in Kenya, Kenyans have built a thriving digital ecosystem encompassing businesses, human progress, problem-solving, innovation, cultural expression, and dynamic social interactions—making their data invaluable for AI training. For instance, a Meta official confirmed that the company has scraped all public data posted on its platforms since 2007 for training AI.⁸ Only users in the European Union are granted an opportunity to opt out of this violation.⁹ In contrast, the publicly provided personal data of all Meta users in other regions has proved an invaluable resource for the company.¹⁰ Beyond social media data, AI systems have also been trained on vast datasets from books, music, and global news archives, often without the creators’ consent.¹¹ This widespread use of data raises serious ethical concerns around intellectual property rights, exploitative data extraction, and the reinforcement of biases in AI models. Ethical AI development depends on inclusive, well-regulated data governance mechanisms that ensure representativeness and accountability and foster responsible innovation.¹²

In that regard, this blog explores unethical data sourcing as it has manifested both globally and locally. It also looks at what entails ethical data sourcing, while considering the dilemma presented by open data initiatives. By examining these issues, the article aims to consider actionable strategies for ethical AI development in Africa through ethical sourcing and governance of data.

2. How Unethical Data Sourcing has Manifested so far Globally

OpenAI, arguably the most recognisable AI company in the world, has faced its latest battle over how it sourced data to train the GPT models. The company is now entangled in a copyright lawsuit filed by Indian book publishers, joining a growing list of legal challenges from authors, news agencies, and musicians worldwide.¹³ These cases highlight ongoing concerns over AI models training on proprietary content without authorisation, threatening intellectual property rights and undermining traditional revenue streams. The lawsuit in India, led by the Federation of Indian Publishers, demands that OpenAI either secure proper licensing agreements or delete datasets containing copyrighted material, reflecting broader global anxieties about the ethical sourcing of AI training data.¹⁴

Other global companies that have faced this challenge include Stability AI, sued by Getty Images for scraping millions of copyrighted photos; Anthropic, targeted by music publishers over song lyrics; and Microsoft, dragged into OpenAI-related suits due to its partnership, all underscoring a mounting tension between AI innovation and the rights of content creators across industries and borders.¹⁵ Stability AI’s legal woes began in February 2023 when Getty Images accused it of using over 12 million images to train its Stable Diffusion model, claiming this unauthorised use fueled a generative AI that competes with Getty’s stock photo business.¹⁶ Anthropic, founded by ex-OpenAI researchers, faced a lawsuit in October 2023 from Universal Music Group and others, who alleged that its Claude model was trained on copyrighted lyrics, and that it often produces almost verbatim reproductions of copyrighted materials, without proper licensing.¹⁷ Meanwhile, companies like Nvidia¹⁸ were hit with a 2024 authors’ lawsuit over pirated books. Perplexity AI, sued by Dow Jones and the New York Post¹⁹ for repurposing news articles, further illustrate how the AI industry has relied on vast datasets sourced from what they term “publicly available information”, which is constantly clashing with established intellectual property frameworks. These developments have raised significant questions about fair use, the sourcing of training data for AI development, the future of the creative industry, and intellectual property considerations.

3. In Africa?

Like much of Africa, Kenya is experiencing extensive data extraction, often in ways that raise ethical concerns. Past research by CIPIT indicates that AI developers require vast amounts of data to train models, and much of this data is sourced through methods like web crawling and scraping. The authors note that while some datasets, such as the Demographic and Health Survey (DHS) and the Malaria Indicator Survey (MIS), are publicly available, others—like detailed reports from the Senegal National Malaria Control Program—may be copyrighted or sensitive but still end up being used without proper authorisation. Many African countries, including Kenya, have copyright laws that do not explicitly address AI training, leaving room for exploitation. Kenya’s Copyright Act allows for fair dealing in scientific research but does not clarify whether AI model training falls under this exception.²⁰ In contrast, South Africa’s Copyright Amendment Bill attempts to define fair use,²¹ though its impact on AI development remains uncertain. This lack of clear legal and ethical frameworks means AI developers often extract data without consent or transparency, which undermines privacy and erodes trust in AI systems. This unchecked data sourcing leads to unethical AI systems—trained on data acquired through questionable means, violating individual rights and reinforcing existing societal inequities.²² Without stronger data governance, the continued extraction of African data to train AI will likely continue to be based on a foundation of exploitation rather than fairness and accountability.

Further exploitation in data sourcing has positioned Africa as a low-wage data annotation hub, where global tech companies outsource AI training tasks. In Kenya, young, unemployed workers, often desperate for economic opportunities, are hired by companies like Meta and OpenAI through third-party firms such as Sama, to label vast datasets – images, text, and videos – for minimal pay, sometimes as low as $2 per hour. This work frequently exposes them to disturbing content, including violence and explicit material, to refine AI algorithms used worldwide. The unethical nature of this sourcing is evident in the economic vulnerability it exploits, offering little job security or mental health support despite the psychological toll. Global companies bypass stricter labor laws in wealthier nations, exploiting Kenya’s weaker regulations.²³ Workers have brought legal action against Meta and Sama, citing long hours, precarious contracts, and a stark power imbalance that limits their rights and avenues for redress.²⁴ Allegations of retaliation against workers attempting to unionise further highlight the exploitative conditions.²⁵ This case exemplifies the digitisation of global inequality, where countries like Kenya serve as low-wage labour centres for AI advancements,²⁶ yet see little of the economic benefits despite their critical role in fueling tools like OpenAI’s ChatGPT ²⁷

Kenya is taking steps to curb the longstanding exploitation of its citizens’ data for AI training by foreign entities. The Ministry of Information, Communications, and the Digital Economy (MICDE) has acknowledged Kenya’s vulnerability to data exploitation, where international companies harvest local data with little to no benefits returning to the population.²⁸ To counter this, the government has emphasised national data sovereignty through the National AI Strategy, the recently released Diplomat’s Playbook on Artificial Intelligence, and proposed²⁹ data governance legislation. These initiatives aim to halt exploitative data practices and ensure that AI development aligns with ethical standards while empowering Kenyans rather than serving solely foreign interests.

Beyond national policy efforts, equitable data practices must also extend to direct benefit-sharing with affected communities. The implementation of Benefit Sharing Agreements can provide a structured way for African communities to receive fair compensation for their data contributions, as exemplified by the Masakhane Pelargonium case. Here, the Masakhane community in South Africa disputed their inclusion under a traditional authority’s Benefit Sharing Agreements (BSA) with Schwabe Pharmaceuticals, demanding a separate BSA to secure benefits from the harvesting of the Pelargonium sidoides and Pelargonium reniforme plants on their land. This effort was rooted in their assertion of self-representation through a Communal Property Association.³⁰ This case highlights the critical need to move beyond superficial consultations toward authentic engagement that respects community agency and delivers tangible rewards. By embracing benefit-sharing, African communities can transition from being mere sources of raw data to active beneficiaries of the technological advancements their contributions enable.

4. Ethical Data Sourcing

Ethical data sourcing is foundational in developing trustworthy AI systems, weaving together data privacy, informed consent, and a spectrum of digital rights tied to data governance. The performance and legitimacy of AI systems depend heavily on the quality and ethical integrity of their data sources – a principle increasingly enshrined in legal and regulatory frameworks worldwide.³¹ Core ethical considerations, including transparency in AI development, robust data protection, and the assurance of fairness and non-discrimination, are critical to fostering AI systems that uphold integrity.³²

Informed consent, a crucial element to ethical data sourcing, is defined by the GDPR as a freely given, specific, and unambiguous acceptance to data processing,³³ signaled by clear action indications through statements or actions, ensuring the data subject understands how their data is used. This emphasis on informed consent underscores an understanding of how one’s data is handled—a principle mirrored in Afrocentric frameworks. For instance, the Malabo Convention (African Convention on Cybersecurity and Personal Data Protection) requires that data processing rest on the “principle of consent and legitimacy of personal data processing,” aligning with claims that processing is legitimate only with consent, except for legally permitted exceptions.³⁴ Likewise, Kenya’s Data Protection Act mandates that personal data processing should only occur once the data subject consents to such processing.³⁵ These overlapping definitions highlight a universal commitment to empowering data subjects, though their implementation varies across jurisdictions.

Understanding the role of data in the design and deployment of AI systems is critical, particularly in ensuring transparent processes for data collection, storage, processing, and governance.³⁶ Bias in AI often originates from multiple sources – how data is collected, the structure of algorithms, and patterns of user interaction. These biases can lead to unequal and discriminatory outcomes, which are then embedded in machine learning models.³⁷ Tackling such bias requires a multifaceted approach, including curating more representative datasets, using bias-aware algorithms, and incorporating robust user feedback mechanisms. Importantly, bias frequently stems from unrepresentative or incomplete data sources, making ethical data sourcing a foundational step toward building fairer and more accountable AI systems.

5. The Open Data Dilemma

Africa’s scarcity of localised datasets has long hindered the development of AI and data-driven solutions tailored to its diverse populations.³⁸ Efforts to build African-based datasets often stumble into exploitation, with concerns over inadequate consent, lack of fair compensation, and external control dominating the narrative. Open data has emerged as a potential antidote, promising to democratise information access and spur innovation.³⁹ In Kenya, the government’s Open Data Initiative (KODI), launched in 2011, was hailed as a pioneering effort to make public data such as census figures and budget data, openly available to fuel transparent development.⁴⁰ Open data proponents see it as a way to empower local developers and reduce reliance on proprietary or unethically sourced datasets, a critical step toward ethical AI.

Yet, the reality in Kenya reveals significant challenges in maintaining government-initiated open data. Updates to the portal have stalled; although by 2016 the portal boasted 849 datasets,⁴¹ the 2022 Global Data Barometer highlighted significant deficiencies in Kenya’s open data initiatives, pointing to outdated and poorly managed data, alongside inadequate capacity that risks exposing sensitive information, ultimately undermining public trust.⁴² In other instances, datasets are often outdated or incomplete, undermining their utility for AI innovation.⁴³ Additionally, privacy concerns further complicate the landscape of open data in Kenya. Publicly released datasets such as health or demographic records, raise fears of re-identification, especially in tight-knit communities where ensuring anonymization is more complex.⁴⁴ Furthermore, questions surrounding data ownership present additional challenges. The issues of who controls data derived from communal resources and how the benefits are shared remain largely unresolved, highlighting the necessity for clearer legal and policy frameworks.

6. Conclusion

As Africa advances toward ethical AI development, robust data governance within AI regulation offers a critical framework essential in balancing innovation, inclusivity, and privacy. Addressing exploitative data extraction, enforcing consent-driven collection practices, and ensuring that AI development benefits local communities are critical steps in this process. Open data, while a catalyst for local AI solutions, requires robust governance safeguards to prevent misuse and inequity. By embedding equitable data-sharing agreements, strengthening regional collaboration, and prioritising transparency, Africa can chart a path toward ethical AI development that empowers its people and ensures that technological advancements align with ethical principles and digital sovereignty. Such an ethical AI policy, grounded in strong data governance, not only drives responsible AI but also empowers Africa’s people, setting a global standard for fair benefit sharing and accountability.