OpenAI Launches New Division to Prevent AI Catastrophes

(iexclusivenews) – OpenAI, the company behind ChatGPT, a popular AI-powered chatbot, is taking steps to ensure the safety and security of its artificial intelligence (AI) systems.

The company announced on Oct. 25 that it is creating a new division called “Preparedness” to assess and mitigate the potential catastrophic risks of AI.

What is Preparedness?

Preparedness is a new team within OpenAI that will focus on tracking, evaluating, forecasting and protecting against the worst-case scenarios of AI misuse.

The team will specifically address the following AI threats:

– Chemical, biological, radiological, and nuclear (CBRN) threats: How can AI be used to create or deploy weapons of mass destruction?

– Individualized persuasion: How can AI be used to manipulate or influence individuals or groups of people?

– Cybersecurity: How can AI be used to hack or disrupt digital systems or networks?

– Autonomous replication and adaptation: How can AI be used to create or evolve new AI systems without human oversight or control?

The Preparedness team will be led by Aleksander Madry, a professor of computer science at MIT and a leading expert in AI robustness and security.

The team will try to answer questions such as:

– How dangerous are frontier AI systems, which are more advanced than the current state-of-the-art models, when put to misuse?

– How likely are malicious actors to obtain or deploy stolen AI model weights, which are the parameters that define how an AI system works?

– How can we prevent or respond to potential AI catastrophes in an effective and ethical way?

“We believe that frontier AI models, which will exceed the capabilities currently present in the most advanced existing models, have the potential to benefit all of humanity,” OpenAI wrote in a blog post. “However, these models also pose increasingly severe risks.”

The company added:

“We take seriously the full spectrum of safety risks related to AI, from the systems we have today to the furthest reaches of superintelligence. […] To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness.”

MORE NEWS:

5 Free ChatGPT and Generative AI Courses to Level Up Your Skills

GPT-4: The Ultimate Guide to OpenAI’s Latest and Most Advanced AI Chatbot

How to Join Preparedness?

OpenAI is looking for talented and passionate people with diverse technical backgrounds to join its Preparedness team.

The company is also launching an AI Preparedness Challenge for catastrophic misuse prevention, offering $25,000 in API credits to its top 10 submissions.

The challenge is open to anyone who wants to contribute to the mission of preventing AI catastrophes.

The participants will have to propose and implement solutions for detecting, preventing, or mitigating potential AI threats using OpenAI’s tools and platforms.

The deadline for submitting proposals is Nov. 30, 2023. The winners will be announced on Dec. 15, 2023.

OpenAI previously announced its plans to form a new team dedicated to addressing potential AI threats in July 2023.

Why is Preparedness Important?

The risks associated with artificial intelligence have been widely discussed and debated by experts, policymakers, and the public.

Many people fear that AI has the potential to become more intelligent than any human and pose an existential threat to humanity.

Despite acknowledging these risks, companies like OpenAI have been actively developing new AI technologies in recent years, which has in turn sparked further concerns.

In May 2023, the Center for AI Safety (CAIS), a nonprofit organization dedicated to ensuring the safe and beneficial use of AI, released an open letter on AI risk.

The letter urged the global community to prioritize the mitigation of the risks of extinction from AI alongside other societal-scale risks, such as pandemics and nuclear war.

The letter was signed by over 200 prominent researchers, academics, entrepreneurs, and activists from various fields and disciplines.

The letter stated:

“We believe that humanity faces an urgent challenge: ensuring that increasingly capable artificial intelligence (AI) systems are aligned with our values and do not pose unacceptable risks.”

The letter also called for more research, collaboration, transparency, and regulation in the field of AI safety.

OpenAI’s Preparedness initiative is a welcome step in this direction. By creating a dedicated team to address the potential catastrophic risks of AI,

OpenAI is demonstrating its commitment and responsibility to ensure that its AI systems are safe and beneficial for humanity.