🤖 AI Learns Computer Use

Anthropic and Microsoft Give AI New Practical Powers

October 24, 2024

Hello AI Enthusiast,

AI just got a lot more hands-on this week. Microsoft is rolling out AI agents that can handle your daily tasks in Office, while Anthropic's Claude can now use computer software by looking at screenshots. It's like watching two tech giants take different paths to the same destination: making AI truly useful in our daily work.

Speaking of being truly useful, we have some exciting news of our own. After months of preparation, we're launching something special that will help you stay ahead of AI's rapid evolution and make AI work for you (Can't wait? Get a sneak peek →).
But more on that later – first, let's dive into this week's biggest AI developments...

The Big Picture 🔊

Anthropic's Claude Gets a Serious Upgrade

Anthropic has announced plans to release Claude 3.5 Haiku - a smaller but powerful model - and rolled out an update to Claude 3.5 Sonnet.
The standout feature? Claude can now use computers by looking at screenshots and figuring out what to do – think of it as having a very eager intern who can handle repetitive tasks for you. The model is scoring well on benchmarks, though it is still far from human-level performance.

Gianluca Mauro
Gianluca MauroCEO and AI Rockstar

Instead of trying to work with countless different APIs, Claude now learns from screenshots – basically the same way humans learn new software. This is not the most efficient approach, but definitely the fastest scalable, as UIs tends to be more consistent that APIs. However it doesn’t work out of the box, and developers needs to define its field of action through - ironically - APIs.


Giving a model access to computer screenshots opens up a lot of concerns about data privacy. Anthropic will have to implement safety gurdrails to contain sensitive information processing.

Microsoft's Bet: AI Agents Are the New Apps

Microsoft is taking a bold step in the AI race with a game-changing announcement: Copilot Studio will soon let anyone create their own AI agents. Plus, they're rolling out ten ready-made agents for Dynamics 365 to help with sales, service, finance, and supply chain tasks. It's basically a team of AI assistants that already know how to use your Microsoft tools.

Gianluca Belloni
Gianluca BelloniCMO and Marketing Nomad

Instead of just throwing powerful AI tools at businesses and saying "figure it out," they're making it surprisingly simple. These agents will work seamlessly within Microsoft's ecosystem - which is an advantage but also the biggest challenge – they need to keep all that sensitive business data safe and secure.


We're seeing two different approaches: while Anthropic's new computer control feature aims to work with any software through screenshots, Microsoft is focusing on making their agents work perfectly within their own ecosystem. It's quality versus quantity, and both have their merits.

Meet the AI Academy Membership

With Microsoft rolling out AI agents and Anthropic pushing the boundaries of AI capabilities, one thing is clear: the AI landscape isn't just changing – it's accelerating.
But here's the challenge: how do you stay on top of these changes and actually put them to work?

That's exactly why we're launching the AI Academy Membership.

  • Unlimited Generative AI Project Bootcamp Cohorts: Join live sessions for every cohort (6x a year), so you’ll always be in sync with the latest AI advancements.

  • 5 AI Certifications: Expand your expertise, including member-exclusive courses like Generative AI for Business Innovation and AI Career Compass.

  • Bi-monthly Live Calls: Gain insider industry insights from AI experts.

  • 1-on-1 Coaching: Direct access to AI experts who speak human.

  • Community and Support: Get 1-year access to our Discord community and AI Academy knowledge chatbot—wherever your questions arise, we’ll be there to help.

  • And much more

This isn't just another course. It's your ticket to riding the AI wave, not just today, but every day. We're starting a movement. And we want you to be part of it.

Bits and Bobs 🗞️

  • NotebookLM has removed its "Experimental" label and introduced new features, allowing users to customize Audio Overviews by guiding AI hosts on focus and expertise level. It will also soon introduce paid tiers for companies, going from a research project to a paid product, just like ChatGPT did 2 years ago.

  • Runaway announced Act-One, a new tool that transforms video performances into expressive animations using just a simple camera setup, eliminating the need for complex motion capture technology.

  • Perplexity has launched new features for Internal Knowledge Search and Spaces for enhanced research and collaboration, allowing users to search both web content and their internal files in one platform.

  • News Corp's Dow Jones and the New York Post have filed a lawsuit against Perplexity, accusing it of illegally copying their content for its search engine and causing financial harm.

  • The Lenfest Institute, in collaboration with OpenAI and Microsoft, is providing grants and enterprise credits to select U.S. news organizations to explore AI technologies for enhancing local journalism sustainability and innovation.

That's a wrap on our newsletter! Here’s a quick recap before you go:

Catch you next week! 👋