Is AGI Actually Scary? Existential Risks From Artificial General Intelligence Explained

Why Trust Techopedia

Can you imagine an AGI robot wandering into your kitchen, locating the coffee maker, finding the coffee beans, and brewing a perfect cup of coffee without being specifically programmed to perform any of those tasks? This scenario, the Coffee Test, devised by Apple co-founder Steve Wozniak, encapsulates the essence of artificial general intelligence (AGI).

What is AGI, and how does it differ from AI?
Unlike traditional AI, which is programmed to execute predefined tasks efficiently, AGI represents a level of machine intelligence that mirrors human cognitive abilities, enabling it to perform a wide array of functions with human-like proficiency.

AGI’s prospect is fascinating and unsettling to many, as it conjures images of self-sufficient machines from blockbuster movies capable of thinking, learning, and acting independently of human oversight.

The prospect of AGI becoming a reality is much closer than you might think. But this also raises profound questions about control, safety, and the unforeseen impacts on society. The big question that looms is whether we should be scared of the people and corporations behind AGI or AGI technology itself.

Key Takeaways

  • Experts fear that AGI development could become a tool of corporate giants with little regard for societal impact.
  • There is an increasing mistrust of big tech firms directing the future of AGI.
  • Zuckerberg promises transparency to democratize AGI development. But many don’t trust him to do this.
  • Elon Must warns of risks AGI will bring to humanity while talking about brain chips and driverless trucks.
  • AGI raises concerns that big tech cannot solve on their own.

Artificial General Intelligence News & Tech Billionaire’s Concerns

Musk’s Warning: AGI Development & the Risk to Humanity

Elon Musk’s recent lawsuit against OpenAI presents a stark narrative of deviation from an original non-profit mission, raising alarms over the direction of AGI development. Initially founded to advance AGI as a benevolent force under open-source and non-profit principles, OpenAI is accused of morphing into a profit-driven entity under Microsoft’s substantial influence.

Musk argues this approach betrays foundational commitments to developing AGI technologies openly and equitably, specifically, technologies that should prioritize human welfare over corporate profitability. Musk’s allegations highlight a concerning transition from a philanthropic vision to a model where AGI development could become a proprietary tool of corporate giants, potentially sidelining the broader good.

This lawsuit underscores legal and ethical concerns and prompts a broader industry introspection about the future trajectory of AGI. The contention that OpenAI’s GPT-4 model might already represent AGI adds another layer of complexity, challenging the industry to define and recognize AGI amidst its profound potential implications.


Musk’s personal and competitive motives weave through the narrative, suggesting that the lawsuit could also be a strategic move against a major rival influencing the competitive landscape in AI.

On April 11, Elon Musk announced that his artificial intelligence firm, xAI, will open-source its chatbot, Grok. Making Grok open-source could allow developers to contribute to the project and accelerate Grok’s adoption. However, Musk has acknowledged that it might take a while before xAI reaches the level of OpenAI.

The outcome of the Musk vs. OpenAI legal battle may well set precedents for how AGI is developed, commercialized, and regulated, potentially reshaping how these powerful technologies are aligned with humanity’s broader interests and safeguarded against exploitation.

Although Musk’s AGI concerns are refreshing, we must also remember that his companies want to implant a computer chip in your brain and deploy fleets of autonomous trucks that drive themselves from pickup to delivery.

Zuckerberg’s Vision for Open-Source AGI

Meta’s infamous CEO, Mark Zuckerberg, has positioned his company at the forefront of developing AGI, emphasizing an open-source strategy.

Zuckerberg’s vision involves merging Meta’s AI research group, FAIR, with the generative AI product team, highlighting a strategic shift to leverage AI breakthroughs across Meta’s extensive user base.

This integration reflects a broader ambition within the tech industry, where companies like Google and OpenAI also vie to unlock AGI’s potential — often termed the pinnacle of AI technology capable of mimicking human intelligence across a spectrum of tasks.

Yet, Zuckerberg’s approach uniquely advocates for transparency, aiming to democratize AGI development amidst a fiercely competitive landscape dominated by offers of high salaries and substantial resources to attract scarce AI talent.

View this post on Instagram


A post shared by Mark Zuckerberg (@zuck)

Zuckerberg believed by being open-source, we have a better chance of mitigating the risks associated with highly advanced AI systems becoming overly concentrated within a few powerful entities. This philosophy sets Meta apart in an industry grappling with the dual challenges of innovation and control of next-generation AI technologies.

By planning to integrate AGI capabilities into everyday applications through devices like smart glasses, Zuckerberg envisions a future where AI-enhanced digital interactions become commonplace, enhancing the utility and engagement of Meta’s platforms.

This strategic pivot underscores a significant commitment to advancing AI technology and addressing the profound societal impacts and ethical considerations it entails.

However, Zuckerberg’s track record suggests we would be foolish to trust the words of a man at the helm of a company that once reportedly told advertisers that it could identify the moment teens felt ‘insecure’ and ‘worthless.’

Once again, these previous examples of saying one thing while doing another intensify AGI fears and make people nervous about big tech’s plans for AGI and its impact on our collective future.

Global Calls for Strict Regulation to Prevent AGI Abuse

The Open Letter

The open letter signed by many industry stalwarts in March 2023 reflected the fear and apprehensions towards the uncontrolled development of AGI.

The letter strongly urges all AI labs to immediately pause the training of AI systems more powerful than GPT-4 until a robust framework is established to control misinformation, hallucination, and bias.

Indeed, the so-called ‘hallucination,’ inaccurate responses, and the bias exhibited by AI solutions on many occasions are too glaring to ignore.

The open letter is signed by Steve Wozniak, among many other stalwarts. It already has over 33,000 signatories, including software developers, engineers, CEOs, CFOs, technologists, psychologists, doctoral students, professors, medical doctors, and public school teachers.

It concludes:

“Humanity can enjoy a flourishing future with AI. Having succeeded in creating powerful AI systems, we can now enjoy an ‘AI summer’ in which we reap the rewards, engineer these systems for the clear benefit of all, and give society a chance to adapt. Society has hit pause on other technologies with potentially catastrophic effects on society. We can do so here. Let’s enjoy a long AI summer, not rush unprepared into a fall.”

The Critical Need for Robust Ethical Guidelines & Regulatory Measures

Furthermore, governmental bodies are investigating the compliance of these AI technologies with stringent data protection regulations like the GDPR, emphasizing the critical need for oversight in safeguarding sensitive and personal data.

The fear surrounding the abuse of artificial general intelligence AGI is palpable, mainly if control falls into the hands of a select few nations or corporations, potentially leading to unprecedented scales of bias and manipulation.

The prospects of AGI acting as a tool for sophisticated information warfare or as a means to dominate sensitive global information pools highlight a grim potential future.

This underscores the critical need for robust ethical guidelines and regulatory measures to ensure that the development and deployment of AGI technologies prioritize the collective benefit over individual or nationalistic gains. This would prevent a future where AGI’s immense potential is misused for destructive ends.

If AGI is merely a reflection of human intellect and potential, should we ask ourselves whether it is genuinely scary, or is it the people behind its creation and use that we should be wary of?

One thing is for sure: the stakes got much higher if Silicon Valley dared to continue to follow its mantra of moving fast and breaking things.

The Bottom Line

The impact of AGI is still up for grabs, but it leaves more questions than answers.

Who knows how it works? Who decides how it works? Who benefits? Who is disadvantaged? Who decides who benefits and who is disadvantaged? Who has the power to stop its development? Who has the power to regulate its development? Who has the power to enforce such regulation? Who holds those who build it accountable? Who holds those who use it accountable? Who holds those who own it accountable?

If big tech is serious about calming the fears of AGI, we need to understand how we can collectively steer the course. Only then do we have a fighting chance of avoiding the pitfalls of past innovations, ensuring they serve the greater good rather than amplifying existing inequalities.


Why is AGI so dangerous?

Why are people afraid of AGI?

What is the wrong side of AI?

Why should people not fear AI?


Related Reading

Related Terms

Neil C. Hughes
Senior Technology Writer
Neil C. Hughes
Senior Technology Writer

Neil is a freelance tech journalist with 20 years of experience in IT. He’s the host of the popular Tech Talks Daily Podcast, picking up a LinkedIn Top Voice for his influential insights in tech. Apart from Techopedia, his work can be found on INC, TNW, TechHQ, and Cybernews. Neil's favorite things in life range from wandering the tech conference show floors from Arizona to Armenia to enjoying a 5-day digital detox at Glastonbury Festival and supporting Derby County.  He believes technology works best when it brings people together.