• AI Academy
  • Posts
  • 📚 Spot AI Search Limitations in Your Research

📚 Spot AI Search Limitations in Your Research

How we tested AI search tools and found their biases

December 03, 2024

Reading Time: 5 minutes

Hello AI Enthusiast,

In this edition, we're looking at something interesting we noticed while using different AI search tools. We'll show you how ChatGPT, Perplexity, and Google can give different results for the same search, and what this means for your daily work.

The Problem

When searching for information online, you might be missing out on useful results depending on which AI tool you're using. We noticed this while looking for AI courses, each search tool showed quite different options. Our founder Gianluca wrote about this in more detail on LinkedIn.

Let's see how different tools handle the same search, and how you can use this knowledge to make your researches.

Our Experiment

We used the same search intent across all platforms:

  • For AI tools (ChatGPT and Perplexity): "I need to find a course about generative AI that can help me build projects without having to know how to code. My goal is to boost my career and build useful things."

  • For Google: "generative AI course with project"

Let's look at what each platform showed:

ChatGPT search results

  • Exclusively recommended courses from large organizations (Google, NVIDIA, DeepLearning.AI, University of Helsinki)

  • Focused on well-known institutions and platforms

  • Missed smaller, potentially more specialized offerings

Perplexity:

Perplexity search results

  • Focused extensively on one option (DeepLearning.AI's course)

  • Provided more detailed information about that single course

  • Referenced but didn't highlight smaller providers in its sources

Google:

Google search results

  • Showed a mix of providers, both large and small

  • Included specialized offerings like our Generative AI Project Bootcamp

  • Presented results based on both relevance and provider size

What We Learned

AI search tools play it safe. Both ChatGPT and Perplexity heavily favor big names and established institutions, often missing smaller but valuable alternatives. While these tools are still evolving and will likely improve over time, for now it's important to use each one strategically:

  • ChatGPT: Get quick insights about established solutions

  • Perplexity: Deep dive into specific options you're considering

  • Google: Cast a wide net and discover all options, including specialized providers

As AI search tools continue to evolve, staying informed about their capabilities and limitations becomes increasingly crucial. Want to keep learning about AI developments and their practical implications? Join AI Academy's membership for regular insights, hands-on experiments, and a community of professionals navigating the AI landscape together.

Your Turn

Try this: Pick a product or service you need to research and test different search approaches. Start with a general keyword search, then try more specific prompts including criteria like features. Compare results across platforms and pay attention to what each tool prioritizes. Are you seeing mostly big names, or a diverse range of options? Keep track of which tool works best for what kind of information.

Want to get even more practical? Explore hands-on AI learning with AI Academy:

We'll be back with more AI tips soon!