Disclaimer
The posts of this website are based on my own opinion only and do not represent IBM's opinions, strategies or ambitions.
1 Introduction
As the technology behind Artificial Intelligence (AI) develops, different foundation models and downstream applications such as general purpose AI (GPAI) emerge and impact various fields. This development is not only accompanied by benefits, but comes along with a broad range of challenges and limitations that society needs to face. Hence, the EU AI Act takes the spot as the pioneer, trying to integrate various regulations and safety mechanism in order to create a safer environment around AI. In this paper, we will focus on GPAI and foundation model applications in high-risk areas, in particular law, education and healthcare. Furthermore, we will explore the European Union’s Artificial Intelligence Act with its regulations that are currently governing the development and application of AI, critically analyzing possible gaps in the current regulatory framework. Finally, we will discuss whether these regulations are beneficial or impending across multiple social and ecological parameters.
2 General-purpose AI
Although, the definition of Artificial Intelligence is not clearly defined throughout the literature, the term general-purpose AI gains more and more importance. The EU AI Act tries to define these systems in the following way: “General purpose AI systems are AI systems that have a wide range of possible uses, both intended and unintended by the developers. They can be applied to many different tasks in various fields, often without substantial modification and fine-tuning” [1]. As for this unifying definition, GPAI models have a wide range of applications across multiple areas, particularly designed to fulfil distinctive and complex tasks, think abstractly and have the ability to adapt to new situations. GPAI systems are characterized by a large parameter size, their functional coverage, and their capability to develop additional functionalities that differ from the originally intended ones. These characteristics distinguish GPAI from the previously dominant single-purpose or single-model AI systems, leading to a shift in paradigm widely throughout established IT infrastructures and sectors. Providers like OpenAI, Meta, IBM and Google focus heavily on the development of foundation models and GPAIs, indicating the big potential and significance of such technology in the world [2]. One of the most known and leading examples for GPAI is ChatGPT, which is based on the foundation models GPT-3.5/GPT-4. This GPAI system is publicly available and can perform multiple tasks, ranging from answering simple question to solving complex mathematical problems, conceptualizing scripts for content producers and summarizing various texts. All these applications are not intendedly trained through the developer, but are rather possible through the GPAIs capability to be fine-tuned through prompt engineering by the user itself [3].
2.1 Applications in high-risk areas
The European Union’s Artificial Intelligence Act follows a risk based approach in order to govern the regulations of foundation models and GPAI and hence, divides the governance in four different categories, minimal risk, limited risk, high risk and unacceptable risk. For this section, we will consider the domains of law, education and healthcare, which are classified through the act as high-risk areas.
2.1.1 Law
The use cases of GPAI in law are very versatile, affecting non legal-professionals, lawyers and judges with high responsibility. The area of law faces many challenges, e.g. immense workloads for lawyers regarding documental research with an old-fashioned, partially digitalized infrastructure or high costs for people who want to get answers to the simplest questions regarding law. GPAI can help to nullify some of these problems and offer an effective way to help law workers in all stages to reduce the workload [4]. Firstly, GPAI or in this context also referred to as LegalAI, can reduce large-scale redundant work for legal professionals. Tasks like retrieving documents and contracts, case related research and overall understanding of law relations can be nearly completely re placed by LegalAI, helping the legal professionals focus on more important tasks. In countries with Common Law Systems, e.g. U.S., LegalAI can provide similar case matching, comparing current case data with a large database of completed cases from the past. Furthermore, LegalAI provides the framework for legal-judgment prediction, allowing lawyers and judges to predict the judgment result based on factual descriptions and the statutory reports of the case. [5] Secondly, LegalAI can provide zero-shot detection of crimes in the future, by using its capabilities of summarizing recordings and video materials and then identifying the crime through zero-shot reasoning. As for the current state of the technology, the surveillance recording descriptions need to be transcribed manually, but in the future the LegalAI system will automate this process, helping lawyers and law enforcement workers to identify crimes [6].
2.1.2 Education
In education, GPAI systems can provide significant help for educational workers and also students, by automating the process of grading exams and helping to formulate clear instructions for the students, personalizing learning experience and deliver real-time feedback. Intelligent Tutoring Systems (ITS) provide platforms for personalized tutoring, by simulating a tutor and allowing direct one-to-one interactions with the student for more effective and adaptive learning. Such AI systems can align the learning process with the respective pace of the student, to ensure that the students learn and understand a topic before they continue with another. This solves a particular problem of current lecturing strategies like face-to-face teaching, where the pace is created by the lecturer for a whole group of students without taking the actual topic understatement of each student into account. Besides that, these platforms can also provide a foundation for student-teacher collaborations, ranging from simple sharing of lecture materials to direct contact with the respective teacher [7]. Another opportunity to use GPAI systems in education is content creating paired with language translation and accessibility. These systems can assist educators by creating new educational content based on recently updated textbooks through the internet. In addition to that, the previous learning content and the generated one, could be translated in various languages so that this content can be used in different countries [4].
2.1.3 Healthcare
As for the last high-risk that is considered in this paper, single-purpose AI systems with statistical approaches are well established in the field of healthcare and medicine. These models rely heavily on the data, since they are trained through a supervised based approach, and hence, are not very adaptive. In addition to the lack of adaption, the datasets, on which these models are trained on, are imbalanced, e.g. certain races, genders or conditions are not equally represented. This issue leads to biases in the output, resulting in either discriminative or false representations [8]. GPAIs on the other hand, which have the capability to generalize to different use case scenarios, solve this particular problem and benefit both medical workers and patients across various fields in healthcare and medicine [9]. In the direct interaction with patients, GPAI systems could support the physicians by answering certain questions from patients, suggesting diagnosis based on the factual descriptions of the symptoms and being available all the time. This reduces the workload from physicians and the waiting-time of patients significantly, allowing physicians to focus on more complex tasks and ensuring that patients can get their desired treatment in time. Their capability to continuously learn through each input they receive, allows the model to improve successively by incorporating the input of professionals throughout the procedures and treatments. This fact is a substantial difference to the prior single-purpose AI systems, since they usually have to be trained from scratch to adjust to certain new developments [9]. Furthermore, GPAI systems, including the foundation models, could solve the issues of AI being not explainable due to the large parameter and feature space of prior established AI approaches. Especially in the medical field, where false decision could harm patients crucially, explainability and understanding are substantial to incorporate the decisions of the AI system. This problem decreases the trust towards the decisions that the AI system suggests, so GPAI and foundation models, being far more explainable, counteract this issue and increase the trust in their suggestions overall [8].
2.2 Challenges and Limitations
Although, the development of AI, in particular recent achievements for GPAIs, has many benefits and applications, it comes along with reasonable challenges and limitations that have to be considered. The increasing functionality of GPAI in various fields and their capability to adapt to new scenarios, exceeding the intended purposes, create a variety of problems that cannot be ignored.
2.2.1 Data Privacy and Safety
The most fundamental risk regarding AI is data privacy and safety. As any other technology that is available online, GPAI systems are vulnerable to cyberattacks. Through these cyberattacks, hackers could potentially get access or reconstruct data, on which the foundation model of the GPAI system was trained on. This data could contain private or sensitive information and copyrighted data, which should not be available publicly [7]. Furthermore, such data is often crawled online and the acquisition of the explicit consent in order to use the data, is in most cases very complex and therefore, not often addressed by the providers [10]. There are techniques such as data anonymization that are applied in order to protect sensitive data, but these methods are not bulletproof, creating the risk of private data being reconstructed through the AI system itself. This fact also aggravates the compliance with data protection laws like GDPR, since the reveal of private data is a violation of all existing legal frameworks [10].
2.2.2 Racial Bias and Discrimination
General-purpose AI systems are build on foundation models, which are usually trained on large datasets. These datasets could potentially contain certain historical biases regarding social stereotypes, inequalities and discrimination, that could be translated into the model’s output. Especially in healthcare and medicine, this could lead to substantial inequalities in treatment strategies and diagnostics based on ethnicity, race or gender [11]. Besides the bias through the data, another source is the algorithm behind the GPAI system or foundation model. Since these algorithms are written, evaluated and tuned by humans, there is a bias due to subjectivity in various development stages of the general purpose AI or foundation model[11].
2.2.3 Abuse of GPAIs capabilities
Another fundamental problem is the abuse regarding the intended purpose of GPAI systems. Foundation models and general-purpose AI systems are designed by the developers in order to accomplish a set of tasks, but their capability to adapt to different scenarios and use cases, creates room for unintended use cases. These unintended applications can be categorized in two classes. One class considers exploitation of vulnerabilities of such AI models and the other one, AI-enabled attacks. Examples for both categories would be social engineering attacks, where the models imitate certain individuals through their writing style or voice, allowing abusers to perform phishing attacks on companies and individuals. Another typical example of such abuse are Deepfakes, which include convincing video and voice generation of individuals, typically famous and powerful people, in order to spread fake statements and influence economics or politics [12].
2.2.4 Environmental Footprint
Another challenge is based on the climate change and how we humans affect the climate change. Although, foundation models and GPAI systems can help humans to reduce their environmental footprint by conceptualizing solutions with greenhouse gases reduced in many regions, the own environmental footprint of these AI systems can not be ignored anymore. As the field of AI develops, the size of the data, infrastructure and models increases continuously. With the exponentially increasing size, the energy consumption and CO2-emission increase as well, e.g. Google’s machine learning Model Meena already consumes energy which is equivalent to an average passenger vehicle driving 242,231 miles for training only [13]. Another metric regarding climate is the water footprint. For AI systems the water consumption can be directly (e.g. maintain the operability of AI systems through cooling) or indirectly (e.g. water-based electricity production), and is involved in production, operation and maintenance of such AI models [14].
3 EU AI Act
As in the previous section displayed, there are not only benefits through GPAIs, but also reasonable challenges and limitations that need to be considered. This is exactly what the European Union’s Artificial Intelligence Act tried to accomplish. As the first regulatory framework across the world, the EU AI Act tries to define certain regulations and rules for companies that want to publish and use GPAI systems based on a risk-based approach. This approach splits the application areas in for different categories: unacceptable risk, high-risk, limited risk and minimal risk. For each risk area, the GPAI systems have to comply to certain regulations and rules. The first submitted proposal for the EU AI Act was submitted in April 2021 by the European Commission. The final agreement on the first proposal was published in December 2023, where the EU Commission and EU Parliament agreed consensually on the proposed regulatory framework.
3.1 Current regulations
The EU AI Act considers foundation models and general-purpose AI systems separately. Foundation models are the base for general-purpose AI systems and need to comply to global restriction, since they are typically more complex and potentially create more risk, based on their powerful capabilities for downstream applications. The regulations for GPAI systems on the other hand are based on the risk-class that they get addressed to. Such models and their technical documentation, purpose and training data will be provisioned and undergo a risk assessment, after which they will be assigned to a particular risk-category. Considering high-risk areas, foundation models need to comply with transparency obligations such as broad testing, documentation and human evaluation so that requirements like safety, cybersecurity and predictability are ensured. Furthermore, the data that is used to train such foundation models needs to comply with copyright law in order to minimize violations and risks of abuse and racial biases. If such foundation models are going to be published on the market as a foundation for GPAIs or other downstream applications, these models need to be registered through the developer on specialized EU databases. After the publication, the complete technical documentation need to be passed to the downstream application developer and a continuous collaboration between both parties needs to be ensured throughout the whole development cycle of GPAI system. If the foundation model also exceeds the average performance regarding complexity, computational power or overall capabilities, this model is classified as ”high-impact”. Being high-impact, additional requirements need to be complied to, regardless of the risk-classification that has been abducted previously [15].
3.2 Gaps in regulations
Although, the regulatory framework of the EU AI Act seems very versatile and considers many scenarios of use cases, it still contains certain gaps and loopholes. One of the biggest issues is the definition of AI, since it includes also statistical approaches. Due to that, simple Excel spreadsheets with statistical calculations would be considered AI and, depending on the field that they are used in, could be either banned or regulated heavily. Another issue results through the given FLOP (number of parameters of foundation model) threshold. The threshold of 1023 FLOP appears to be too high, since only the foundation model GPT-3 surpasses this parameter size. Based on this fact, only GPT-3 and higher models are considered ”high-impact” although, models with far fewer parameters perform similarly good as GPT-3+, e.g. Mixtral 8x7B or Llama 2 [16]. Additionally, incorporated exclusion of open-source foundation models can also be considered as a particular loophole in the EU AI Act. It certainly has benefits regarding the freedom of developing such open source foundation models, counteracting the formation of monopoles in the industry and overall empower the AI ecosystem. But considering models that perform similarly good as closed source models like GPT-4, the risk of abuse overweight these benefits [17]. A different concern is raised through the risk assessment itself. The whole regulatory framework is based on the risk classification of the foundation and general-purpose AI models, but the concrete explanations of how providers should assess such risk is not addressed. This leads to two issues. On the one side, providers who try to comply with the act struggle to assess their risk correctly, due to the complexity and costs of the assessment process. On the other hand, the lack of precise guidance offers companies, who do not want to comply to certain regulations, the opportunity to do so, based on an incorrect assessment [18]. Furthermore, the exclusion of open-source foundation models from the global regulations, such as transparency, can be considered two-sided as well. On the one side, this exemption could help to develop open-source models through smaller companies and re searchers. Besides that, this counteracts the monopolization of such technology on the market through hyperscalers like Google DeepMind, OpenAI or Meta. But on the other side, this freedom also creates severe risks regarding the abuse of open-source models. As from the current technological state, some open-source models perform as good as closed-source models (e.g. GPT-4), but are excluded from regulations completely. But such open-source foundation models could be even more dangerous than the regulated closed-source models, since their public access to the source code allows individuals to potentially remove security layers and perform harmful actions to society[17]. Lastly, since the EU AI Act not only considers systems developed in the European Union but also exterior AI systems that are used in the EU, the current version of the regulatory framework could lead to technological segregation. The regulations could hinder innovations of foundation and general-purpose AI models developed in the EU. Additionally, the mandatory compliance of AI systems of Non-EU developers and providers, could make it unattractive to these providers to deploy and offer their services in the EU. These two points combined could have the consequence that EU will fall back in the AI competition, making it either dependent on other countries in this regard.
4 Discussion
As in this paper displayed so far, the topic of general-purpose AI is two-handed, as it offers many opportunities for humankind but at the same time creates risk and challenges that cannot be ignored. The regulation approach of the EU AI Act seems to be very substantial but still needs to be developed, since it contains gaps and loopholes counteracting the effort to have a unified regulatory framework for the development and application of GPAI/foundation models. The current version of the EU AI Act will most likely show significant impact on companies in the near future, but the long-term consequences, whether they are social or economic based, are not particularly addressed. Additionally, this lack of knowledge is also amplified by the fact that there are no other similar regulatory frameworks in the whole world. But this uniqueness raises also a different question: Why aren’t other countries trying to regulate this fast-growing and impactful technology? Quite the oppose, they are heavily investing in developing and researching foundation models, generative AI and general-purpose AI system. The European Union seems to be the only government that neither invests in AI and its development nor encourages entrepreneurs to invest in it. Even worse, this very strict regulatory framework, especially considering foundation models, which are the backbone of every GPAI system, hinders innovation and without additional investment, it is most likely that the EU will not be able to compete in this field. As in the previous chapter displayed, the exclusion of open-source models from the regulations, unless they are considered high-risk, raises concerns as well. The European Union should think about including open-sourcing in their regulatory framework, at least on a fundamental basis, to balance innovation and public safety. This could be accomplished through hosted access to the models, in order to monitor their usage. In addition to that, the FLOP benchmark needs to be either adjusted or replaced by more versatile and independent benchmarks and stress tests, since only one foundation model architecture (GPT-3 and higher) surpasses this threshold [17]. Finally, the regulations as for high-risk areas should be extended regarding safety protocols and cybersecurity to the limited and minimal risk area as well. Currently, only transparency obligations are applied to these two areas, although the risk of abuse is equally represented [17].
5 Conclusion
The current development state of general-purpose AI and their foundation models with their capabilities open many opportunities across multiple sectors, e.g. law, education and healthcare. But although these opportunities can be driving the ecosystem and sectors forward, it also leaves room for risks and dangers such as discrimination, abuse and data safety. Therefore, the EU AI Act seems to be the correct approach to address the issues and to regulate this developing technology. However, the current version of the EU AI Act can be only considered as a first solution to this problem due to the gaps and loopholes that are still present. The regulatory framework excludes open-source foundation models and GPAI systems if they are not classified as high-risk. Besides that, the act is not providing any guidance for the risk assessment, but assigns the providers and developers to do it regardless of the system. These were only an example, but all the mentioned concern need to be considered for future versions of the EU AI Act. In addition to the issues with the EU AI Act, investments should also be considered by the EU, in order to not fall behind and getting dependent on technologies from other countries.
References
[1] Future of Life Institute. General Purpose Ai and the AI Act. May 2023. url: https: //artificialintelligenceact.eu/wp-content/uploads/2022/05/General
Purpose-AI-and-the-AI-Act.pdf.
[2] Connor Dunlop Sabrina K¨uspert Niclas Mo¨ es. The value chain of general-purpose
AI. 2023. url: https://www.adalovelaceinstitute.org/blog/value-chain general-purpose-ai/.
[3] Isaac Triguero et al. General Purpose Artificial Intelligence Systems (GPAIS): Prop
erties, Definition, Taxonomy, Societal Implications and Responsible Governance.
2023. arXiv: 2307.14283 [cs.AI].
[4] Rishi Bommasani et al. On the Opportunities and Risks of Foundation Models. 2022.
arXiv: 2108.07258 [cs.LG].
[5] Haoxi Zhong et al. How Does NLP Benefit Legal System: A Summary of Legal
Artificial Intelligence. 2020. arXiv: 2004.12158 [cs.CL].
[6] Anj Simmons and Rajesh Vasa. Garbage in, garbage out: Zero-shot detection of
crime using Large Language Models. 2023. arXiv: 2307.06844 [cs.CL].
8
[7] Firuz Kamalov, David Santandreu Calong, and Ikhlaas Gurrib. New Era of Artifi
cial Intelligence in Education: Towards a Sustainable Multifaceted Revolution. 2023.
arXiv: 2305.18303 [cs.CY].
[8] Shaoting Zhang and Dimitris Metaxas. On the Challenges and Perspectives of Foun
dation Models for Medical Image Analysis. 2023. arXiv: 2306.05705 [eess.IV].
[9] M. Moor et al. “Foundation models for generalist medical artificial intelligence”. In:
(2023).
[10] Zhongxiang Sun. A Short Survey of Viewing Large Language Models in Legal Aspect.
2023. arXiv: 2303.09136 [cs.CL].
[11] Anja Thieme et al. “Foundation Models in Healthcare: Opportunities, Risks &
Strategies Forward”. In: Extended Abstracts of the 2023 CHI Conference on Hu
man Factors in Computing Systems. CHI EA ’23. ¡conf-loc¿, ¡city¿Hamburg¡/city¿,
¡country¿Germany¡/country¿, ¡/conf-loc¿: Association for Computing Machinery,
2023. isbn: 9781450394222. doi: 10.1145/3544549.3583177. url: https://doi. org/10.1145/3544549.3583177.
[12] Ta´ ıs Fernanda Blauth, Oskar Josef Gstrein, and Andrej Zwitter. “Artificial Intelli
gence Crime: An Overview of Malicious Use and Abuse of AI”. In: IEEE Access 10
(2022), pp. 77110–77122. doi: 10.1109/ACCESS.2022.3191790.
[13] Carole-Jean Wu et al. “Sustainable AI: Environmental Implications, Challenges and
Opportunities”. In: Proceedings of Machine Learning and Systems. Ed. by D. Mar
culescu, Y. Chi, and C. Wu. Vol. 4. 2022, pp. 795–813. url: https://proceedings. mlsys.org/paper_files/paper/2022/file/462211f67c7d858f663355eff93b745e
Paper.pdf.
[14] A. Shaji George, A. S. Hovan George, and A. S. Gabrio Martin. “The Environ
mental Impact of AI: A Case Study of Water Consumption by Chat GPT”. In:
Partners Universal International Innovation Journal 1.2 (Apr. 2023), pp. 97–104.
doi: 10.5281/zenodo.7855594. url: https://puiij.com/index.php/research/ article/view/39.
[15] Jure Globocnik. The European AI Act. url: https://www.activemind.legal/ guides/ai-act/.
[16] Lianmin Zheng et al. Chatbot Arena Leaderboard Week 8: Introducing MT-Bench
and Vicuna-33B. 2023. url: https://lmsys.org/blog/2023-06-22-leaderboard/.
[17] Philipp Hacker. What’s missing in the EU AI Act. url: https://verfassungsblog. de/whats-missing-from-the-eu-ai-act/.
[18] TechBriefly Newsroom. The European Union’s Ai Act: Pros and cons for the future.
url: https://techbriefly.com/2023/12/11/the-european-unions-ai-act pros-and-cons-for-the-future/.