What is Ethical AI? Preventing bias and discrimination

AI Ethics is a set of regulations that guide the output of artificial intelligence. The advent of Big Data has driven companies to favor automated processes and making data informed decisions, with the goal of improving business results, which has resulted in unexpected situations while applying some of their AI, mostly due to inadequate research design and biased datasets.

The human factor in ethical AI cannot be overemphasized as humans come with all kinds of bias, which are displayed in human behaviors then in our data. It is crucial when testing data and building algorithms to bare this in mind, as artificial intelligence possess the tendency to increase these human biases at an unprecedented rate. Bias and discrimination are not only present in human resources; they can also be found in a variety of applications ranging from facial recognition software to social media algorithms.

Binns & Gallo (2019) using a hypothetical scenario (below) to illustrate a ML system that can lead to discrimination and bias as part of ICO AI auditing.

‘‘A bank has developed a ML system to calculate the credit risk of potential customers. The bank will use the ML system to approve or reject loan applications. To train the system the bank has collected a large set of data containing a range of information about previous borrowers, such their occupation, income, age, and whether they repaid their loan. During testing, the bank wants to check against any possible gender bias and finds the ML system is giving women lower credit scores, which would lead to fewer loans being approved.’’

Two main reasons were given for this, (i) imbalanced training data and (ii) training data reflecting past discrimination.

i. Imbalanced training data imbalance proportion of male vs female in the data can be the source of the bias, as the training data may include a higher population of male loan applicants because the bank doesn’t have enough female data due to low number of female applicants in the past. This bias will affect any sub-population under-represented in a training data e.g., data including ethnicity, skin color.

ii. Training data reflecting past discrimination Biased based on past discrimination, an instance where female loan applicants were denied often, using a training data with past discrimination can lead to repetition of the same pattern. Industry with similar discrimination history such as recruitment can repeat this pattern even if it’s not planned by the designer.

ML is powerful because it discovers hidden patterns in data, the same pattern that exhibit discrimination.

Mitigating bias and discrimination in ML models

In the case of imbalanced training data, offsetting the data by adding more data to compensate for the inadequate female data or removing from the male data. Another approach is to separate the data based on gender and train them separately. Though using different training data separated by gender can be said to be discrimination itself.

When training data reflects prior discrimination, it is best to adjust the data/model or change the learning process.

Other approach on mitigating bias include:

Anti-classification — which opines that a model is fair if protected features or its proxies are removed from consideration.

Outcome and error parity — a model is only fair if it gives equal no of outcome (positive or negative) to separate factions and if it generates a similar number of errors to both groups.

Equal calibration — a model is fair if the predicted outcome matches the actual event. It doesn’t necessarily imply good calibration, as long as both groups have same imperfections.

These are just a few of the ways that can be used to understand and mitigate bias and discrimination in machine learning. Organizations should consider approach best suited to their domain and document each step of any AI application for the purpose of effecting the appropriate safeguards and technical measures.

In the attempt to assist businesses, develop AI that operates fairly and is ethical, the European Commission published a set of guidelines. They laid out seven key areas to focus on to build trust in AI development, they include human agency and oversight, safety, privacy and data governance, traceability, transparency, diversity, non-discrimination and fairness, societal and environmental impact, and accountability.