India's Supreme Court cracks down on AI-generated fake court orders

The Supreme Court of India has taken a strong stance against the use of artificial intelligence (AI) to generate fake or non-existent judicial orders, declaring such practices to be misconduct rather than simple errors in decision-making. This strong pronouncement came after a trial court in Andhra Pradesh relied on AI-generated judgments that were later found to be fabricated, sparking significant concern over the integrity of the judicial process. A bench comprising Justices P.S. Narasimha and Alok Aradhe observed that a decision based on such non-existent and fake alleged judgments has a direct bearing on the integrity of the adjudicatory process and issued notices to the Attorney General, Solicitor General, and the Bar Council of India to examine the consequences and accountability. Senior advocate Shyam Divan has been appointed as an amicus curiae to assist the court in this matter. The case originated from a property dispute where a trial court dismissed objections to an advocate commissioner's report, citing four Supreme Court judgments. However, it was argued that these cited judgments were non-existent and AI-generated. While the Andhra Pradesh High Court acknowledged that the judgments were AI-generated and issued a caution, it proceeded to decide the case on merits, affirming the trial court's order. This led the aggrieved parties to appeal to the Supreme Court. The Supreme Court's intervention highlights a growing concern about the unchecked use of AI in legal proceedings. In a separate hearing in February 2026, a bench led by Chief Justice Surya Kant had also expressed alarm over lawyers using AI tools to draft petitions, citing non-existent judgments like "Mercy vs Mankind" and fabricating quotations from real cases. This trend, the court noted, compromises the accuracy and integrity of court filings. The judiciary's increasing reliance on technology, while aimed at enhancing efficiency and reducing case pendency, faces significant ethical challenges. 'AI hallucinations,' where AI generates plausible but false information, pose a substantial risk in a domain where accuracy and precedent are paramount. The Supreme Court's White Paper on Artificial Intelligence and Judiciary had previously identified 'Fabrication of Cases and Hallucination' as a primary risk associated with AI. Legal experts and judicial bodies are increasingly emphasizing the need for rigorous verification of AI-generated outputs. While AI can assist in legal research and administrative tasks, it cannot replace human judgment, critical analysis, or ethical reasoning. Policies, such as the one adopted by the Kerala High Court, mandate meticulous human verification of any AI-generated legal content and prohibit its use as a substitute for judicial decision-making. The Supreme Court's firm stance underscores the critical need for a robust legal and ethical framework governing the use of AI in the Indian judiciary. The court's objective is to examine the broader implications of AI misuse, ensuring that technological advancements uphold, rather than undermine, the principles of justice and accountability. The legal consequences for those found to be misusing AI in judicial processes are expected to be significant, serving as a deterrent against such practices. The recent legislative reforms in India, including the Digital Personal Data Protection Act and new criminal law bills, signal a proactive approach towards regulating AI and its impact on various sectors, including the justice system. These reforms aim to establish accountability, ensure transparency, and safeguard fundamental rights in the evolving landscape of artificial intelligence.

Frequently Asked Questions

Read Full Story on Quick Digest