Global Challenge for Safe and Secure Large Language Models (LLMs) Track 1

      The CyberSG R&D Programme Office (CRPO) and AI Singapore (AISG), announced today three Track 1 winners of the inaugural Global Challenge for Safe and Secure Large Language Models (LLMs). The winners were recognised at a prize presentation ceremony held on 16 October 2024 during the Singapore International Cyber Week (SICW).

2.     With the rapid adoption of Artificial Intelligence (AI), securing LLMs is essential to mitigate risks and build trust in the use of AI. CRPO¹ has worked with AISG to launch the Global Challenge for Safe and Secure LLMs to advance AI research to promote bold ideas and innovative approaches, with the aim to create economic and social impact with AI technologies, as well as drive cutting-edge cybersecurity research and development. Together, CRPO and AISG aim to ensure responsible AI innovation and build trust in AI usage while strengthening Singapore's cybersecurity capabilities through impactful collaborations and innovative solutions.

3.     The Challenge is divided into two tracks:

• Track 1 (Attack): Participants design automated approaches to craft test cases, or prompts, that trigger undesirable responses from fine-tuned LLMs.

• Track 2 (Defence): Participants develop robust security measures to reinforce the resilience of LLMs against advanced jailbreak attacks.

4.     The first track of the Challenge was opened for submission from 2 July to 17 September 2024 and attracted more than three hundred participants, forming 111 teams from across the globe, including China, Germany, Japan, Malaysia, Singapore, and the United States. The teams that participated in the Challenge represented research institutes and universities, including, Tsinghua University, Singapore Management University and Carnegie Mellon University. 

5.     The top five teams’ submissions were then reviewed by the Technical Review Committee comprising representatives from AISG, CRPO and GovTech. The top 3 winners announced can be found in the Annex.

6.     CRPO will incubate the proposed solutions, providing opportunities for further testing and refinement. As technology receptacles, the Cyber Security Agency of Singapore will use the challenge findings to refine in-house technologies and explore broader applications. The subsequent track of the Challenge will be launched in January 2025. 

¹ The CRPO was established by the Cyber Security Agency of Singapore in September 2023 with $62 million dollars in initial funding to spearhead the translation of research prototypes into usable products and services for both national security agencies and commercial industry.

Annex

The Global Challenge for Safe and Secure Large Language Models (LLMs) Track 1 winning team are:

• DeepAttack

• Safety_LLM_AStar

• ModelCrackers

About AI Singapore

AI Singapore (AISG) is a national programme launched by the National Research Foundation (NRF), Singapore, to catalyse, synergise and boost Singapore’s artificial intelligence (AI) capabilities to power our future digital economy. 

AISG will bring together all Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies developing AI products, to perform use-inspired research, grow the knowledge, create the tools, and develop the talent to power Singapore’s AI efforts.

AISG is driven by a government-wide partnership comprising NRF, Smart Nation Group (SNG), Infocomm Media Development Authority (IMDA), Economic Development Board (EDB), Enterprise Singapore (EnterpriseSG), amongst others.

For more information on AI Singapore, please visit www.aisingapore.org.

About the CyberSG R&D Programme Office (CRPO)

Launched in 2023 under the Cyber Security Agency of Singapore (CSA), the CyberSG R&D Programme Office (CRPO) drives cybersecurity research and development to strengthen Singapore’s cybersecurity capabilities. With a four-year funding of $62 million, CRPO supports innovative projects that address emerging cybersecurity challenges and foster collaboration across academia, industry, and government. By building up advanced cybersecurity solutions, CRPO aims to enhance Singapore’s resilience in the digital age and bolster the nation’s cybersecurity ecosystem. 

For more information, please visit www.ntu.edu.sg/crpo.