When Data Becomes Power: An Anti-Oppressive Approach to AI in Social Work
Dr. Marina Badillo-Diaz, LCSW
Jessie Holston, MSW
The Data That Powers Human Service Organizations
Artificial intelligence is increasingly influencing how social workers and human service organizations document services, evaluate programs, identify patterns, allocate resources, and make decisions. As these technologies become more common, much of the conversation has focused on whether AI systems are accurate, ethical, or biased. Before we can fully answer those questions, however, we must examine something more fundamental: the data that powers them.
Data is not neutral. Numbers and datasets do not offer purely objective, value-free reflections of reality. They are shaped by human choices at every stage. People and institutions decide what will be measured, how categories will be defined, who will collect the information, how it will be interpreted, and what will be left out. Those choices reflect organizational priorities, cultural assumptions, historical conditions, and existing power structures. When data is used to train or guide an AI system, these decisions do not disappear. Instead, they become embedded within the technology and may influence decisions at a much greater speed and scale.
This article brings together two complementary areas of social work expertise: data and evaluation and the ethical integration of artificial intelligence. Jessie Holston, MSW, Founder and Principal Consultant of Kwelanga Minds & Measures, LLC, brings nearly 15 years of experience in behavioral health, community-based services, data analytics, evaluation, and social-impact initiatives. Her work helps organizations move beyond viewing data solely as a compliance requirement and instead use it to understand communities, identify inequities, and create measurable and sustainable change.
Dr. Marina Badillo-Diaz, LCSW, Founder of The AI Social Worker, focuses on the ethical, equitable, and responsible integration of AI into social work practice and education. Through teaching, scholarship, training, and consultation, her work examines how social workers can engage emerging technologies while maintaining human judgment, professional ethics, and close attention to power and inequity.
Our work intersects around a central premise: AI cannot be separated from data, and data cannot be separated from the people, communities, and systems that produce it. Jessie’s work asks organizations to look beyond the numbers and recognize the human experiences and structural conditions behind them. Marina’s work asks social workers to look beyond an AI-generated output and critically examine how the technology arrived at its conclusions. Together, these perspectives raise an urgent question: What happens when data shaped by unequal systems becomes the foundation for increasingly powerful AI systems?
From Data to AI: Where Power Enters the Process
For data to become part of an AI system, it typically moves through several stages. Information is collected, organized into categories, analyzed or used to train a system, and then used to generate content, make predictions, or inform decisions. Human choices are present throughout this process.
Someone determines which information is worth collecting. Someone defines the categories into which people and experiences will be placed. Someone decides what counts as a positive outcome, a warning sign, a service need, or evidence of progress. These choices may appear technical, but they are also social, political, and ethical.
Consider a behavioral health organization that measures attendance, hospitalization, crisis-service utilization, and program completion because those variables are readily available in its administrative records. These indicators may provide useful information, but they cannot tell the entire story. A person’s sense of belonging, trust in a provider, cultural safety, experience of dignity, or definition of progress may remain invisible because those experiences are more difficult to quantify. If the organization relies only on the information that is easiest to collect, its data may provide an incomplete and potentially misleading picture of effectiveness.
The same problem becomes more consequential when these records are used to train an AI system. The system does not independently recognize that important experiences are missing. It identifies patterns in the data it receives. If the available data reflects only a narrow institutional definition of success, the system’s outputs will likely reproduce that definition.
When Historical Inequity Becomes Training Data
Data reflects the systems from which it is produced. In social work and human services, those systems have been shaped by racism, poverty, ableism, unequal access to care, over-policing, surveillance, and other structural inequities. AI can reproduce these patterns even when it has not been explicitly programmed to discriminate.
For example, hospitalization rates and crisis-service utilization are sometimes used as indicators of client risk or program success. Yet a community with limited access to consistent outpatient behavioral health services may have higher rates of crisis contact because emergency services are among the few options available. Similarly, a community that has been over-policed or disproportionately monitored may show higher levels of system involvement, not because its residents are inherently more dangerous or higher risk, but because the system is more likely to observe, report, and respond to their behavior.
If an AI model treats these patterns as characteristics of individuals rather than as consequences of unequal systems, it can reinforce the original harm. A person may be classified as higher risk based partly on historical patterns created by inadequate resources, disproportionate surveillance, or discriminatory institutional practices. In this way, AI can mistake the consequences of oppression for predictors of future behavior.
This is why identifying “bias” in AI is not enough. Bias is often discussed as though it were an isolated technical defect that could be removed with a better algorithm or a more diverse dataset. An anti-oppressive approach requires us to ask deeper questions: Who created the conditions represented in the data? Who defined the categories? Whose understanding of risk or success is being treated as authoritative? Who benefits when the system’s conclusions are accepted, and who carries the consequences when those conclusions are wrong?
Who Gets Represented—and Who Becomes Invisible?
Not everyone is represented equally in data. Communities that are underserved by health, education, and social-service systems may also be underrepresented in the datasets used to inform those systems. In other situations, communities may be heavily represented through records of surveillance, crisis involvement, or institutional intervention while their strengths, relationships, cultural knowledge, and everyday experiences remain undocumented.
Standardized categories can also erase complexity. Rigid classifications of race, gender, family structure, disability, or service outcomes may fail to capture the ways people understand themselves and their communities. When individuals must fit their experiences into categories developed without their input, the resulting data may reflect institutional convenience more accurately than lived reality.
This raises an important question about whose knowledge is recognized as data. Administrative records and numerical measures often receive greater legitimacy than client narratives, qualitative interviews, community knowledge, and practitioner observations. Yet behind every data point is a person whose experience is more complex than the variable used to represent them. If AI systems are trained primarily on information that institutions already value, other forms of knowledge may become even less visible.
An anti-oppressive approach does not reject quantitative data. It challenges the assumption that numbers are inherently more objective or meaningful than lived experience. Ethical analysis requires us to examine what the data reveals, what it obscures, and what it was never designed to capture.The Risks and Pitfalls of Data-Driven Decision-Making
The risks of dataism emerge when data is treated as neutral, complete, or superior to relational knowledge. Social service data is often shaped by historical bias, unequal surveillance, and structural oppression. Communities that are most heavily policed or monitored generate the most data, which can then be used to justify further intervention and control.
Predictive tools can also create feedback loops, where past inequities are projected into the future. When algorithms are trained on biased data, they can reinforce racialized, class-based, and ableist assumptions about risk and deservingness. Without transparency and accountability, these systems make it harder for social workers and service users to question or contest decisions that affect their lives.
From Data Subjects to Data Partners
Organizations collect data about communities without meaningfully involving those communities in determining how the information should be used. People become data subjects rather than partners in the creation of knowledge. An anti-oppressive approach requires a shift from collecting information about communities to developing knowledge with them.
Community members and clients should have meaningful opportunities to help determine what is measured, how categories are defined, which outcomes matter, and how findings are interpreted. They should also have a voice in deciding when AI is appropriate, when it should be limited, and when it should not be used at all.
This partnership requires organizations to value qualitative knowledge and lived experience alongside quantitative measures. It also requires data minimization. The fact that information can be collected does not mean that it should be collected. Organizations should be able to explain why particular information is necessary, how it will be protected, who will have access to it, and how its collection benefits the people providing it.
This commitment to minimization sits in tension with another goal of equity-centered work. Identifying disparities often requires collecting sensitive information such as race, ethnicity, language, gender identity, or disability status, because we cannot see who is being underserved without it. Yet collecting that same information creates risk if it is poorly protected, used for surveillance, or defined without community input. The answer is neither to collect everything nor to collect nothing. It is to ask, for each data element, what question it helps answer, who decided that question mattered, and who controls the data afterward. Frameworks from Indigenous data sovereignty, such as the CARE (Collective benefit, Authority to control, Responsibility, and Ethics) Principles for Indigenous Data Governance, offer useful models for shifting control of data from institutions to the communities it describes (Global Indigenous Data Alliance [GIDA], n.d.).
Meaningful consent and transparency are equally important. Clients and communities should understand when their information is being used by an AI-enabled system and how that system may affect services or decisions. They should also have accessible ways to question, correct, or challenge AI-supported conclusions. Human review should not be treated as a symbolic safeguard; it must involve professionals who have the authority, knowledge, and willingness to override a technological recommendation.
The Social Worker as a Critical Data and AI Practitioner
Social workers do not need to become data scientists to engage critically with data and AI. However, data and AI literacy are increasingly becoming part of ethical social work practice. Practitioners must be able to recognize both the capabilities and limitations of the technologies influencing their work.
The profession’s commitments to dignity and worth of the person, client self-determination, cultural humility, social justice, and professional competence do not disappear when technology enters the room. Those values should shape how social workers evaluate a dashboard, risk score, AI-generated summary, or automated recommendation. A system may be efficient while still undermining self-determination. It may appear accurate while relying on categories that erase cultural differences. It may produce a polished summary while omitting the context needed to understand a client’s experience.
Imagine a case manager uses an AI tool to summarize six months of progress notes before a treatment team meeting. The summary accurately lists missed appointments and a positive drug screen. It leaves out that the client lost her housing during that period, took on care for a family member, and told her case manager she stopped coming because a staff member had spoken to her disrespectfully. Nothing in the summary is false. However, without that context, a story about a client navigating barriers and a broken trust becomes a story about disengagement, and the treatment plan that follows may respond to the wrong problem.
Critical practitioners resist the passive acceptance of technological outputs. They ask who is represented in the data and who is missing. They examine how the data was produced, what assumptions are embedded in its categories, and who defined the desired outcome. They consider who may benefit if an AI-supported recommendation is followed and who may be harmed. Most importantly, they ask what clients and communities know that the available data cannot tell them.
These questions should not be asked only after harm occurs. They belong in conversations about technology procurement, program design, organizational policy, staff training, and evaluation. Social workers should be involved before an AI system is purchased or implemented, not simply after it begins influencing practice.
When a specific tool is being vetted for use, these questions become practical asks. Before your organization adopts one, consider asking your leadership:
What decision will this tool inform or make, and who acts on its output?
How will we monitor its impact across racial, cultural, disability, and other groups, and what would trigger a pause or discontinuation?
Where does client data go, who can access it, and can the tool's developer use it to train other products?
Asking these questions early is not obstruction. It is how social work's values get a seat at the table before the technology does.
Reclaiming Data as a Tool for Liberation
Recognizing the risks associated with data and AI does not mean rejecting them. Data can also be used to make inequity visible, document service gaps, challenge unfair resource allocation, strengthen advocacy, and demonstrate community strengths. The issue is not whether data carries power, but how that power is used and who is able to exercise it.
In fidelity review and quality-assurance work, strong data can support what frontline staff have been saying for years but have struggled to demonstrate. It can help organizations make the case for additional funding, increased staffing, different services, or changes to policies that are not meeting community needs. Data can transform individual observations into evidence of a broader pattern that decision-makers can no longer easily dismiss.
Consider a program where staff repeatedly report that clients miss appointments because of unreliable transportation and changing school or work schedules. Told one at a time, these reports can be dismissed as anecdotes. However, a fidelity review, a structured check of whether a program is being delivered as designed and producing its intended results, might show that missed appointments cluster at particular sites, times of day, and neighborhoods. The pattern changes the question from, "Why aren't clients engaging?" to "What would it take to make this service reachable?" and gives leadership evidence for requesting transportation support, flexible scheduling, or additional staff.
Good documentation and reliable data can also protect programs and clients. They may help a program maintain funding, respond to an unfair audit, or demonstrate a client’s progress in ways that affect services and opportunities. In these situations, data becomes more than a tool for measurement. It becomes a tool for protection and advocacy.
Aggregated information can also reveal patterns that individual caseloads cannot. A case manager may understand the experiences of the people they directly serve, while program-level data can show whether particular interventions are effective across a larger population or whether a system is consistently failing a specific group. It allows organizations to see the forest without losing sight of the individual trees.
AI may help identify patterns that would otherwise remain hidden, but those patterns still require interpretation. Technology cannot independently determine whether a pattern reflects individual behavior, institutional practice, structural inequality, or some combination of the three. Human judgment, community knowledge, and ethical safeguards remain essential.
Data literacy is therefore a form of power. When social workers and communities can understand, question, and interpret data, they are better positioned to challenge dominant narratives rather than simply be defined by them. Data can move from numbers to insight, from insight to understanding, and from understanding to equitable change.
The Question Is Not Just What the Data Says
Data is not neutral, and neither are the systems built from it. AI raises the stakes because it allows data to shape judgments and decisions with unprecedented speed, reach, and authority. Without critical examination, an automated system can give historical inequities the appearance of technological objectivity.
An anti-oppressive approach requires social workers to look upstream. Rather than focusing only on an AI system’s final output, we must examine the people, institutional priorities, categories, omissions, and power structures that shaped its data. We must also ensure that communities have a meaningful role in deciding how data and AI will be used in decisions that affect their lives.
The goal is not to reject data or AI. It is to ensure that social workers and communities possess the knowledge and power necessary to question, interpret, and use these tools in ways that advance equity rather than reproduce injustice.
Reclaiming data’s power for good does not mean the difficult questions disappear. It means social workers are better equipped to ask those questions and act on the answers. The question for social work is no longer simply, What does the data tell us? We must also ask: Who created the data? Whose experiences does it represent? Who gets to interpret it? And who gains power when we act on it?
References
Global Indigenous Data Alliance. (n.d.). The CARE principles for Indigenous data governance. Retrieved September 19, 2026, from https://www.gida-global.org/careprinciplesgida-global
AI Disclosure Statement
The content in this blog was created with the assistance of Artificial Intelligence (AI) and reviewed and edited by Dr. Marina Badillo-Diaz, LCSW, and Jessie Holston, MSW, to ensure accuracy, relevance, and integrity. Dr. Badillo-Diaz's and Jessie’s expertise and insightful oversight have been incorporated to ensure the content in this blog meets the standards of professional social work practice.