AI GOVERNANCE, CORPORATE ACCOUNTABILITY AND REGULATORY COMPLIANCE: EMERGING CHALLENGES FOR COMPANIES IN INDIA

Abstract

AI (artificial intelligence) is arguably one of the greatest technological developments of the 21st century and has already had major impacts on corporate governance structures, corporate decision-making, compliance systems and commercial operations. Increasingly, various sectors within the Indian economy are using AI to improve efficiencies in their operations including recruitment, e-commerce, customer service, finance such as tax planning or financial analysis, risk assessment, cybersecurity and data analytics. However, AI also raises many complicated legal and regulatory issues including those related to accountability and responsibility, transparency and protection of individual privacy, protection of individuals from discriminatory practices and business liability. The lack of comprehensive legislative regulation for AI in India creates uncertainty for companies/organizations that seek to balance the need for innovation with the need to comply with existing laws/regulatory compliance requirements.

The paper addresses the evolving structure of AI governance in India and highlights the challenges faced by businesses regarding assurance of accountability and compliance. This includes a review of laws currently in effect including the Companies Act 2013, the Information Technology Act 2000, Digital Personal Data Protection Act 2023 and various industry sector-specific laws, to see how applicable these existing legal structures are in the Indian regulatory context. Additionally, global developments, particularly the European Union AI Act are analyzed to evaluate their relevance to the Indian regulatory environment. The conclusion of this paper is that traditional corporate governance structures are inadequate to address the risk posed by autonomous AI systems and algorithmic decision-making processes. It promotes the implementation of a risk-based AI governance framework with a focus on ethical accountability, human monitoring, transparency and regulatory scrutiny. 

Keywords: Artificial Intelligence, Corporate Governance, Regulatory Compliance, Corporate Accountability, Digital Personal Data Protection Act, India.

1. INTRODUCTION

   
   

Artificial Intelligence (AI), which began as a technological advancement, is rapidly becoming one of the most essential tools used by modern businesses today. In addition to automating various business functions, AI is being utilized by businesses throughout the world to increase their engagement with customers, optimize supply chains, predict future consumer behaviour and make better overall business decisions. The manner in which AI has been adopted in India has increased significantly over the last few years due to accelerated digital transformation initiatives, greater levels of investment in technology and continued government support for innovation. Nevertheless, continued growth of AI technology has resulted in many legal and governance challenges related to its application. For example, issues such as algorithmic bias, lack of transparency in decision-making processes, improperly collecting and using personal data, cyber-security threats and software failure of unmanned systems are all areas of concern with respect to corporate accountability. As such, there is a pressing need for a clearly defined legal framework to regulate the increasing influence that AI will have on employment, financial services, healthcare and public administration. The goal of this regulatory framework should be to provide accountability for the use of AI while not stunting the innovation that comes along with it.

Three issues are explored in this paper: 

1. To what degree does the present Indian legal system adequately provide for overseeing Corporations’ usage of AI? 

2. What barriers do corporations face when performing due diligence regarding the responsible, lawful and ethical use of AI? 

3. What reforms are needed to develop an effective regulatory framework to oversee AI? 

   
   

The purpose of this paper is to better understand the intersection of technology regulation and corporate governance. With increasing reliance on AI as part of corporate strategy, it is vitally important for all stakeholders (policymakers, regulators, corporations, investors and consumers) to understand their legal responsibilities and governing duties associated with AI usage. The paper attempts to inform the larger conversation around ethically deploying AI and the future viability of corporate governance in India.

2. CONCEPTUAL AND LEGAL FRAMEWORK

   
   

Due to the rapid development of AI, the way Companies Act manage and make decisions has changed significantly. AI technology includes machine learning, natural language processing, predictive analytics and generative AI. AI technologies will continue to be incorporated into many aspects of a company’s operations such as recruitment, customer support, financial management, compliance and risk assessment. The main priorities related to the evolution of AI governance globally have been transparency, accountability, privacy, cybersecurity and the ethical ramifications of automated decision-making. 

Regulatory discussions mostly addressed issues like data protection and information safety, however, as AI systems have become more autonomous and powerful, governments and international organizations are now beginning to create dedicated frameworks for the algorithmic accountability of experts along with the human oversight and responsible innovation of AI. The OECD AI Principles, UNESCO Recommendation on the Ethics of AI and the European Union AI Act are examples of frameworks to assist in the development of responsible and innovative policies related to all elements of the life cycle of AI systems. 

AI Governance in India is still an evolving concept. While the Government has been fairly proactive in promoting the use of technology in the country through programmes such as NIFI Aayog’s National Strategy for Artificial Intelligence (AI) programme, no comprehensive legislation has yet been enacted in India that governs AI specifically. Thus, the regulation of AI today is through the use of several existing legislative frameworks. The Information Technology Act of 2000 would apply to electronic transactions and cybersecurity aspects of AI; the Digital Personal Data Protection Act of 2023 establishes parameters for how personal information may be collected, stored and processed; and the Companies Act of 2013 would generally apply to directors and management in fulfilling their obligations of due diligence, oversight and risk management as it relates to the deployment of AI systems. Other applicable laws include: consumer protection laws, competition law and guidelines/regulations issued by regulatory bodies such as the Reserve Bank of India (RBI), Securities and Exchange Board of India (SEBI) and Insurance Regulatory and Development Authority of India (IRDAI). In addition to the above, while these laws provide for some level of protection, they are not adequate or specifically designed to address the unique features of autonomous and algorithmic decision-making systems, which results in significant regulatory gaps.

The scope of scholarly work on AI governance in India has similarly expanded with respect to constitutional rights, data protection, corporate responsibility and digital regulation. There are scholars who investigate how decision-making through AI could be impacted by the fundamental rights of privacy, equality, dignity and freedom under the Constitution of India. Since the Digital Personal Data Protection Act 2023 became law, academic discourse on the relationship between AI technology and data governance has been on the rise. Still, while there is a rapidly growing body of literature addressing issues of ethics in AI and regulation of digital content, there has been limited research into the specific issues surrounding AI governance, corporate accountability, and compliance with regulatory standards in relation to the Indian corporate sector. Such a gap is particularly noteworthy given that businesses are employing AI technologies more and more to carry out tasks that have direct impacts on consumers, employees, investors and other stakeholders. Thus, the need for a thorough review of AI governance from the perspective of corporate law is urgent and necessary to provide an understanding of new and emerging issues regarding regulatory challenges and the developing responsibilities of companies engaged in business in India.

3. LEGAL ANALYSIS 

   
   

3.1 AI Governance and Corporate Accountability

Artificial Intelligence (AI) has revolutionized business decision-making by automating operations such as recruitment, customer service, financial management, compliance monitoring and risk assessments. When automated decisions cause harm or breach legal rights, AI systems raise complicated problems about duty and culpability in contrast to traditional governance models that presume decisions are made by identifiable human actors. As part of Corporate Accountability, companies are responsible for developing, implementing and using AI systems ethically and legally. While AI systems can be designed to operate autonomously, companies that create or own such systems are still responsible for the outcomes of those systems, regardless of whether human interaction was involved in the development and training of the system. Companies can manage potential and actual risks by having effective governance procedures in place, conducting a risk assessment and implementing safeguards will also help to reduce potential risks.

The "black box" problem is one of the most significant barriers to the implementation of responsible AI solutions. The inability to ascertain the rationale behind the automated decision-making process can make it very difficult to determine whether any decision was the result of discrimination, privacy violations or consumer harm. While the companies that employ AI technology are typically the primary entity responsible for the actions taken by the technology, other parties involved in the creation or training of the technology, including developers, providers of data and users of AI technology, share in some degree of responsibility for the actions of AI technologies. Corporate governance requires that Boards of Directors and management implement adequate oversight of AI-related risks through a combination of accountability, transparency, human rights protections, and compliance systems. Companies responsible for implementing responsible AI practices will also use, in the absence of specific AI laws in India, existing data protection, consumer protection and corporate governance frameworks to preserve Trust and Compliance with public and regulatory authorities.

3.2 Statutory Framework Governing AI in India

A. Companies Act, 2013

The Companies Act, 2013 forms the basis for corporate governance in India. Similarly, it has a major effect on how companies will use AI, although the Act does not actually use the term AI as such, many of its provisions detail duties required of directors which also relate directly to AI. Section 166 of the Companies Act requires directors to act with honesty, exercise due diligence, exercise due care, and be mindful of the best interests of the company, as well as its stakeholders. The use of AI systems without proper oversight is likely to constitute a breach of these duties. The Act also provides that directors must ensure the company has adequate internal controls and risk management systems as stated in Section 134(5), which could extend to AI-related risks. Audit Committees as stated in Section 177 of the Act, could also be involved in the oversight of technology related and compliance related risks from the use of AI. Therefore, companies should develop the governance of AI as part of their overall corporate governance and risk management.

B. Digital Personal Data Protection Act, 2023

India’s most critical legislative measure regarding Artificial Intelligence (AI) governance is the Digital Personal Data Protection Act (DPDP Act). Since AI systems make extensive use of personal data for training and running their operations, compliance with applicable data protection requirements will be of utmost importance. The DPDP Act mandates that organizations that process personal data do so legally, obtain informed consent, use personal data only for the purpose for which it was collected, limit the amount of personal data that is collected or processed and provide reasonable security safeguards for the protection of personal data against unauthorized access to and/or misuse of personal data. All organizations that are designated as data fiduciaries will therefore need to ensure that all personal data is processed responsibly and secured against unauthorized access and/or misuse. AI-based technologies, including facial recognition technologies, predictive analytics, personalized marketing and automated profiling will be required to comply with the rules and standards set forth in the DPDP Act. If an organization fails to comply with the DPDP Act, it may be subjected to significant financial penalties and damage to its reputation. 

C. Information Technology Act, 2000

India’s legislation around electronic governance, cybersecurity and digital transactions is primarily governed by the Information Technology Act of 2000. Since AI systems are susceptible to cyber-attacks, data breaches and authorised manipulation, compliance with cybersecurity regulations is imperative. Section 43A of the Act establishes liability for an organisation that does not have reasonable security practices in place that result in a loss (wrongful or gain). The Information Technology (Reasonable Security Practices and Procedures and Sensitive Personal Data Or Information) rules clarify the need for secure data. As such, businesses that employ AI technology must create and implement solid cybersecurity systems to guard against the loss or compromise of data and to ensure the continuity and reliability of their processes. 

D. Consumer Protection Act, 2019 

The Consumer Protection Act was passed in 2019, which has far-reaching effects for companies that utilize AI to develop products or services. Many of these products or services, such as chatbots, recommendation systems, automated customer service platforms and digital marketing tools, are likely to modify the way customers make purchasing decisions as well as how they interact with companies. If a business uses one or more of these AI tools to provide incorrect or misleading information, engage in unfair trade practices or cause injury or harm to a consumer, it is possible the business will be liable for violating consumer protection laws. The consumer protection act makes it clear businesses that use ai to conduct customer facing activities are responsible for all behaviour and outcomes associated with that ai.

3.3 Relevant Judicial Developments

A. Justice K.S. Puttaswamy v. Union of India (2017)

The Supreme Court of India unanimously acknowledged the right to privacy as a basic right under Article 21 of the Constitution in this historic ruling. The Court stated that maintaining individual control of personal data and informational privacy constitute an important aspect of human autonomy and dignity. The Puttaswamy principles establish the constitutional framework for AI governance in India, and likewise, AI systems rely heavily upon the acquisition, processing and analysis of large quantities of personal data. Companies that utilise AI technology will have an obligation to ensure their data practices comply with privacy rights and with legal requirements, such as consent, purpose limitation and data protection.

B. Shreya Singhal v. Union of India (2015)

The Supreme Court struck down Section 66A of the Information Technology Act of 2000 because it deemed that restrictions on speech online should be constitutional and reasonable. The Court also expressed deep concerns over the arbitrary regulation of content online and reiterated the importance of free speech in the digital context. The ruling will affect how Internet platforms use automated decision-making tools and AI-based content moderation systems. The Court’s ruling points to the need for accountability and transparency regarding the use of AI systems that impact online expression and access to information. 

C. Internet and Mobile Association of India v. Reserve Bank of India (2020)

The Supreme Court ruled that the RBI's circular restricting banking services to cryptocurrency businesses was unconstitutional. The Court’s application of the proportionality principle indicated that unless legislation restricts or limits the ability of human beings to innovate or create, there must be clear evidence that restrictions pose harm or risk, therefore, laws should only restrict new technologies such as cryptocurrencies based on a clear and direct relationship to existing laws and/or an identifiable risk. Although this ruling was specific to cryptocurrencies, the rationale applies equally to any future attempts to regulate AI. Therefore, future frameworks for AI governance must balance the competing interests of fostering innovation and regulating such that there is no unreasonable limitation placed on technological advancement while protecting society's interests.

3.4 Doctrinal Analysis

AI's popularity has caused several key legal and governance concepts to emerge: transparency, accountability, fairness, explainability, human oversight and risk-based regulation. These principles attempt to protect the growth of AI in society, by assuring compliance with laws, ethical principles and fundamental human rights in specific sectors, such as employment, finance, healthcare and corporate governance. One major concern about AI governance is that many AI systems behave like a “black box,” generating decisions in a manner that does not clearly articulate how an AI system generates a particular outcome. As such, the notion of explainability becomes even more important for an organization making automated decisions, as it provides support for defending those decisions, while also allowing for greater regulatory scrutiny. Similarly, with respect to human oversight, the individuals and enterprises that design, deploy and monitor AI are ultimately the responsible party for legal liability.

AI governance is about fairness and non-discrimination. Because AI systems rely on past data, they can reproduce existing biases, leading to discriminatory results. To identify and minimize algorithmic bias, organizations should implement auditing, testing and impact assessments. Many governments are implementing transition to a risk-based approach to regulation therefore, governments require greater obligations of their regulated entities when regulating higher risk AI applications than they do for lower risk AI applications. In addition to legal compliance, AI governance incorporates ethical obligations to be accountable for their actions, to protect the privacy of individuals, to protect the rights of individuals, to be transparent and to respect individuals' autonomy in relation to AI systems. These ethical principles provide the basis for ethical AI governance and accountability of organizations.

4. CRITICAL ANALYSIS AND EMERGING CHALLENGES

5. 
   

4.1 Critical Evaluation

India’s regulation of Artificial Intelligence (AI) is presently constraint by sectoral laws and fragmentation, despite India having a relatively progressive regulatory environment for many areas of business, due to laws regulating a company’s ability to operate with AI or other technologies in a certain manner (for example, many laws apply to data protection, privacy and cyber security, and/or the corporate governance of a company). Further, current laws do not sufficiently provide for the regulation of certain AI issues that are often raised by stakeholders as concerns regarding algorithmic bias in AI-generated outputs, AI-based autonomous decision making, and/or the need for specific risk assessments for implementing AI.

The absence of clear rules specifying blame when AI systems inflict damage is one of the major limitations of the current framework. With the increasing use of AI to aid business decision making, determining who is accountable between developers, vendors and the deploying firm is no easy task. Similarly, the lack of laws providing for transparency and the ability to explain oneself makes it nearly impossible for affected individuals to understand or contest an automated decision, resulting in reduced public confidence and accountability. At the same time, India’s comparatively lax regulatory framework has also fostered innovation and enhanced the development of new technologies. If there are regulations that are overly restrictive during the early stages, it may reduce the level of investment that companies would make and will ultimately reduce competitiveness. Therefore, while the existing regulatory framework is not sufficient to adequately respond to the anticipated future issues posed by the introduction of AI, it does create an opportunity to develop a balanced regulatory framework to enable innovation while ensuring transparency, accountability and adequate risk management.

4.2 Competing Views

Opinions vary among experts on how much regulation should apply to artificial intelligence (AI). Some people believe in flexibility and innovation, therefore, they do not want stringent regulations because such regulations might discourage investment and increase the costs to comply, which could slow technological development. They believe that establishing and following voluntary ethical codes of conduct, with companies regulating themselves, encourage innovation while allowing businesses to quickly adapt to new technologies. Others believe that there are some fundamental rights, such as the right to privacy and no discrimination against an individual, that need to be protected through legal means. These experts believe that businesses cannot adequately protect people's rights unless they are required by law.

The concept of risk-based regulation is becoming more popular among stakeholders, who want different levels of regulation depending on the risk associated with the different types of AI systems. In general, they want to determine a higher-level regulatory approach for high-risk AI systems while allowing for a lower-level regulatory approach for low-risk applications. In addition, this regulatory framework will support innovation while providing adequate protections to individuals in society. In India, it is very important to maintain this balance in order to support technical innovation while maintaining the trust of the public and providing corporate accountability.

4.3 Emerging Challenges

Businesses are increasingly using AI to enhance their operations, however, the growing presence of this technology is also causing various ethical, legal and regulatory concerns. One of the major areas of concern is algorithmic bias/discrimination. AI systems trained using historical data may result in discriminatory or biased outcomes for example, when selecting employees, giving loans, offering insurance or rating how well individuals perform in their jobs. Businesses could face all types of legal consequences, reputational damage and/or regulatory challenges as a result of these AI actions. Another major obstacle to an organization's use of AI is data governance and privacy. Organizations need large amounts of data to train their AI systems, thereby creating issues surrounding permission to collect data, protecting data security, developing profiles on consumers and conducting unlawful surveillance. While the Digital Personal Data Protection Act 2023 provides some level of protection, organizations are required to balance innovating through the use of AI while complying with applicable laws in a manner that respects a person's right to privacy. 

Organisations in various industries and geographies face difficulties achieving compliance due to different regulations across multiple sectors and jurisdictions. Compliance with different laws and government agencies can lead to differing programmes, mandates, and/or requirements that can cause overlapping compliance challenges for businesses. In addition, businesses with operations in multiple countries face further compliance challenges when trying to comply with global compliance frameworks such as the EU AI Act. The rapid increase in generative AI and deepfake technology has led to significant concerns regarding the creation of misinformation, competing interests in intellectual property ownership, damage to reputation and the possibility of corporate fraud, which requires organisations to develop flexible and coherent governance frameworks for AI.

4.4 Implications for Corporate Governance

The growing use of AI will have a profound effect on Corporate Governance. Senior management and boards of directors will be tasked with incorporating AI-related risks into corporate governance and risk management systems. In order to ensure the effective governance of AI systems, firms must establish monitoring procedures, assess risks, and ensure that their AI systems are compliant with applicable laws and ethics. Businesses are starting to adopt measures to facilitate the responsible use of AI, including the establishment of AI governance committees, AI impact assessments and AI audits. Stakeholders have become more demanding of accountability and transparency regarding AI-related issues as well as the risks associated with its use. The relationship between AI and ESG (Environmental, Social and Governance) factors has never been more closely linked than today, as ethical decision making, privacy, and justice are enhanced through the use of AI in an ethical manner.

5. CONCLUSION

   
   

Having incorporated artificial intelligence into their operations, businesses are experiencing a significant transformation in the landscape of governing organisations and how they hold each other accountable, as well as regulatory compliance. Although AI will offer many benefits in creativity, efficiency and economic growth all of which require sufficient amounts of privacy or protection from discrimination, a major challenge to business is how to maintain a balance between the use of AI and the privacy and protection of individuals' rights. The legal framework that governs India’s corporations includes relevant laws, such as the Companies Act 2013, Digital Personal Data Protection Act (DPDPA) 2023, Information Technology Act 2000, and various Consumer Protection Regulations. However, the aforementioned laws were not drafted with any intention of addressing the unique challenges posed by AI-assisted decision-making systems. 

Traditional processes of corporate governance are losing their effectiveness as automated technologies exercise considerable control over company decisions. Effective governance requires greater board oversight, transparency, human management and independent audits of AI systems. India needs to therefore implement a comprehensive risk-based regulatory framework for AI that incorporates the principles of accountability, transparency, explainability, fairness and human oversight. Policymakers can draw on global models to modify these principles to India’s constitutional values and development goals. Businesses should take proactive steps to create internal governance and compliance procedures for AI to reduce potential legal and ethical risks. Research should look at the challenges of governing AI in various sectors, the ways in which enforcement can work and the relationship between AI regulation, competition law and emerging technologies. A comprehensive and flexible regulatory framework will be necessary to ensure AI innovation develops in line with principles of corporate accountability, consumer protection, and constitutional law. 

REFERENCES

Companies Act, 2013 (Act No. 18 of 2013).

Consumer Protection Act, 2019 (Act No. 35 of 2019)

Digital Personal Data Protection Act, 2023 (Act No. 22 of 2023). 

Information Technology Act, 2000 (Act No. 21 of 2000).

Information Technology (Reasonable Security Practices and Procedures and Sensitive Personal Data or Information) Rules, 2011. 

Justice K.S. Puttaswamy (Retd.) v Union of India, (2017) 10 SCC 1.

Shreya Singhal v Union of India, (2015) 5 SCC 1.

Internet and Mobile Association of India v Reserve Bank of India, (2020) 10 SCC 274.

Baxi UP, Law and Technology in India (Oxford University Press 2020).

Susskind R, Tomorrow’s Lawyers: An Introduction to Your Future (2nd edn, Oxford University Press 2017).

Brownsword R, Scotford E and Yeung K (eds), The Oxford Handbook of Law, Regulation and Technology (Oxford University Press 2017).

NITI Aayog, National Strategy for Artificial Intelligence #AIForAll (2018).

OECD, OECD Principles on Artificial Intelligence (2019).

UNESCO, Recommendation on the Ethics of Artificial Intelligence (2021).

European Commission, Artificial Intelligence Act (2024).

Hildebrandt M, ‘Accountability and Responsibility in Artificial Intelligence’ (2018) 34 Computer Law & Security Review 299.

Cath C and others, ‘Artificial Intelligence and the Good Society’ (2018) 24 Science and Engineering Ethics 505.

Veale M and Borgesius FZ, ‘Demystifying the Draft EU Artificial Intelligence Act’ (2021) 22 Computer Law Review International 97.

Tiwari A, ‘Artificial Intelligence and Data Protection in India’ (2023) 15 Indian Journal of Law and Technology 45.