An AI that can actually crawl website pages

There is a public website called parcoursup that has a school page for thousands of schools in France. Parcoursup is a public website, no login is required.

I asked Leo AI to crawl all school pages and return to me a list of all schools that meet a certain criteria based on Parcoursup’s public information (such-as : more than 5% of accepted candidates come from out-of-city candidates).

Leo AI immediately replied that it does not have the ability to crawl a website.

Please note that I am not asking for webscrapping, more asking a list of schools that meet a certain criteria.

If Leo AI can’t do this, how can I ask an AI to do this ? Which alternative to Leo would be able to give a shot at my query ?

@ppbe

At the present moment, you would make searches at websites and online, and then create a summary list of your findings - example:

“Schools in Provence - a PDF file at:
[URL address]”

“Normandy - school administration report, [name of report], at:
[URL address]”

You find the sources of info and report those to an AI ← and ask for the summary of what that AI finds when using those sources.

I would ask each AI service:

“Parcoursup is a web portal designed by the French Ministry of Education and the French Ministry of Higher Education, Research and Innovation. Are you able to use that service - its website - to look over all school pages and return a list of all schools that meet a certain criteria based on Parcoursup’s public information (such-as : more than 5% of accepted candidates come from out-of-city candidates)?”

The AI will let you know its limitations.

I have been using Alter.systems and Perplexity.ai.

Hi,

This is precisely the point. I asked a prompt in the lines of what you wrote, and the AI response from Brave was that is was not able to crawl a website, and said I should do it manually myself.

Hence my question, if the AI engine inside Brave can’t do it, then does anyone know of an AI capable of doing it ?

List of AI projects and services:

https://en.wikipedia.org/wiki/List_of_artificial_intelligence_projects

Plenty for you to investigate by asking the question previously mentioned. At Wikipedia, I would scroll down to “Natural language processing” and start interviewing those listed.

The Wikipedia list is not comprehensive - and does not mention:

  • Leo AI
  • Alter.systems
  • Perplexity.ai

So you may want to search around for a more thorough list.

Might interest:

https://i10x.ai/?fpr=aixploria2&el=aixploria2

https://www.aixploria.com/en/ultimate-list-ai/’ ← Apparently, there are at least 5,000 !

Effectively, AI search engines:
https://www.aixploria.com/en/category/search-engine/

If tempted by https://www.genspark.ai/ ← avoid them. Because of a specific Genspark feature - where the AI can place automated phone calls on your behalf.

You could try https://brave.com/blog/ai-browsing/, it gives Leo the ability to visit and go through web pages