Bipartisan Submissions in a Criminal Trial Infected with Hallucinating AI – Twice

In the course of a recent Victorian Supreme Court criminal trial, written submissions made jointly by the prosecution and defence, upon a request by the Judge in respect of a discrete issue, were found to contain fictitious case citations and Parliamentary speeches.

Upon this being discovered, the Judge called for revised submissions, which most unfortunately, contained legislation that did not exist.

On this issue, Justice Elliott said in Director of Public Prosecutions v GR [2025] VSC 490 (14 August 2025) at [61] – [80]:

G. Use of artificial intelligence

61. A further matter needs to be raised concerning written submissions filed with the court. More specifically, the matter concerns the due administration of justice and the use of artificial intelligence in the preparation of submissions or other materials filed or otherwise put before the court.

62. As is apparent from the above reasons, an issue arose in this proceeding on the power of the court to make particular orders under section 24(1) of the Act. In seeking to obtain assistance from the parties on this issue, an email was sent to them on 30 July 2025. That email noted that the materials filed to date did not address the fact that section 24(1) did not expressly provide for a child to be remanded in a facility under the control of youth justice. The parties were invited to provide an outline of submissions on the extent of the power of the court to remand GR at various locations in the event that he was declared liable to supervision under Part 5 of the Act. The email concluded in the following terms:

His Honour requests that, if the parties are able to reach agreement, a written outline of joint submissions on this issue be filed by 4.00pm on Wednesday 6 August 2025. If the parties are unable to reach agreement, then separate written outlines of submissions are to be filed by the same time and date.

63. At 4.54pm on 5 August 2025, my associates received an email from a partner of the law firm acting for GR which stated that the parties had conferred on the issue. The addressees of the email included the prosecution’s counsel and solicitors, and defence counsel. The email “attached submissions as to the joint position agreed by both the prosecution and defence”, and foreshadowed the document being filed. This in fact occurred shortly after. The document filed by GR’s solicitors was entitled “Submissions on Custodial Remand under s 24 of the [Act]” and was not signed by counsel or solicitors for either party.

64. In preparation for the hearing, the court considered these filed submissions. However, it was not possible to locate some of the materials referred to in those submissions, including what purported to be direct quotes from cases recorded as being decisions of this court.

65. Accordingly, my associates emailed the parties and their counsel the morning before the hearing requesting the parties provide copies of those cases, together with the Second Reading Speech and the Commission’s Report that had also been referred to and purportedly quoted.

66. Later that morning, GR’s senior counsel sent an email to my associates stating that the filed submissions were wrong. Further, the email stated that the citations were wrong on the basis that “[t]hey do not exist”. It was further noted that the cases referred to were incorrectly cited and did not apply to this matter. Senior counsel took full responsibility for the errors, apologised and foreshadowed amended submissions being filed in the near future.

67. Early that afternoon, revised submissions filed on behalf of “The Accused” were emailed to my associates by senior counsel. The covering email stated that the prosecution were aware of the contents of the revised submissions and did not seek to make any further submissions. An apology was again made for the errors in the previous document.

68. The revised submissions unsurprisingly made no reference to the non-existent cases. They also removed fictitious quotes from what were previously said to be parts of the Second Reading Speech and the Commission’s Report.

69. An email was then sent by the court noting receipt of the revised submissions. The email acknowledged the initial submissions had been filed by the defence and that GR’s senior counsel had taken responsibility for the state of those initial submissions. However, it was further noted that the submissions had been filed on “a joint basis” and accordingly the parties were directed to provide either a joint explanation or individual explanations as to how those initial submissions had come to have been filed.

70. Later that afternoon, the court received an email from GR’s senior counsel explaining that artificial intelligence had been used to assist with the joint submissions. The court was informed that the initial citations were checked and were accurate and that an assumption was made, wrongly, that the later cases and citations were also correct. On this basis, the submissions were forwarded to the prosecution for its approval. Again, senior counsel took responsibility for the “fault” and made an unreserved apology to the court.

71. The draft submissions that had been forwarded by the defence as referred to in the preceding paragraph were reviewed upon receipt by both counsel and solicitors acting for the prosecution. In an email sent at 4.38pm on 5 August 2025, the prosecution advised defence counsel and solicitors that “we agree with the position expressed in the submissions you provided”.^[31] The email invited GR’s solicitors to file the submissions “noting they were agreed by us”, and expressed appreciation if that were done. The email stated the prosecution’s intention to file written submissions that “speak to the entire process” and stated that those submissions would include the following:

In the present case, we agree with the defence that if the court declares [GR] liable to a supervision order he may be remanded in a youth justice centre under s 24(1)(e), which empowers the court to make “any other order the court thinks appropriate”.

To be clear, there was no suggestion that the prosecution took any issue with the draft submissions that were intended to be filed.

72. In an email sent to the court the afternoon before the hearing commenced, the prosecution explained its position in relation to the “joint submissions document”. The prosecution stated the joint submissions document was not checked “because we ultimately agreed with the conclusion at paragraph 20”. It was further stated that the prosecution should have checked all references in any documents provided. It was acknowledged that the position was unsatisfactory and an apology was also proffered by the prosecution.

73. The pervasiveness of potentially misleading information caused by the use of artificial intelligence did not end there. The revised submissions filed the afternoon before the hearing were not properly reviewed by defence or prosecution counsel.^[32] Indeed, the revised submissions referred to legislation that did not exist, and also a provision in the Act that was said to have been inserted and then repealed, which in fact never occurred and which provision never existed.

74. These matters were raised with counsel at the start of the hearing yesterday. After appropriate apologies were given, it was agreed that GR’s counsel would be given the opportunity to file and serve further submissions that did not contain inaccuracies as a result of misinformation provided by artificial intelligence.

75. Accordingly, it was not possible to conclude the hearing yesterday (as had been expected prior to this issue arising) and the matter resumed today after the court had the benefit of further revised submissions from the defence.

76. At the risk of understatement, the manner in which these events have unfolded is unsatisfactory.

77. Observations have been previously made in other cases about the unsatisfactory consequences that may flow from litigants using artificial intelligence in the preparation of materials to be relied upon for court purposes.^[33] The comments in these cases have been confined to circumstances where self-represented litigants have utilised this technology in an attempt to present their case.

78. In May 2024, this court published “Guidelines for litigants. Responsible use of artificial intelligence in litigation”.^[34] It is essential that all litigants and practitioners adhere to these guidelines.

79. The ability of the court to rely upon the accuracy of submissions made by counsel is fundamental to the due administration of justice. Self-evidently, as was immediately and unequivocally acknowledged by counsel in this case, any use of artificial intelligence without careful and attentive oversight of counsel would seriously undermine the court’s processes and its ability to deliver justice in a timely and cost-effective manner.

80. Regrettable as it is to single out counsel and their instructing solicitors in this case for what has occurred, in light of the matters set out above it is important to record that counsel must take full and ultimate responsibility for any submissions made to the court. To this end, it is not acceptable for artificial intelligence to be used unless the product of that use is independently and thoroughly verified. The same may be said for solicitors responsible for producing or filing court documents.

The link to the full decision is here.