This page is best viewed with a Desktop or Tablet computer

If you have a small screen device it may offer you a choice to switch to a “simple” display. If you accept that choice it will be easier to read the text, but you may miss the images and the important links.

Announcing!

A New App! A Technical Internet Utility

Available Fall 2023! App Testers Wanted Now!

MetaQuest

This is an internet crawling, HTML scraping Android App. Quest throughout the metaverse, crawling from website to website, scraping here and scraping there, to be wherever, to find whatever there is to be found,

to see what you can see…

Celtic dragons, symbolic of power and wisdom,

protect the Earth and all living things.

 

 

MetaQuest: Ethical Internet-crawler & HTML-scraper

Crawl wherever! Scrape whatever!

 

Special Note

MetaQuest is currently in the Open Testing stage where any users can evaluate the app. This post is an open invitation to anyone who visits my website or my Facebook page to be part of the Open Testing Group. More info about this towards the end of this announcement.

 

What  is MetaQuest?

MetaQuest is an Android app that facilitates questing through the internet, following links from website to website, peering in here and there, getting an introduction to ethical internet crawling and HTML scraping, to go wherever you go, to find whatever you may come across, to see what there is to be seen. Yes, this is a true quest through unfamiliar, and perhaps unknown, facets of the MetaVerse. “Meta” means beyond, and a “Quest” is an arduous journey :)

 

As an internet crawler and HTML scraper MetaQuest permits you to visit arbitrary websites and to extract whatever data may be found there - technical bits such as details of the HTML code, images, icon, author, description, keywords, Meta Data, Forms Data, Media, and especially IP addresses, geoLocations and links - and more especially - links to other websites!

 

Unlike a traditional Web Crawler, with MetaQuest the web crawling is under your control. A typical web crawler such as a Google bot is given a list of “seed sites” and turned loose to crawl and scrape. With MetaQuest, you are the bot and MetaQuest is your tool for crawling and scraping. You control the choice of seed site, which sites you will visit and what data you will scrape. For ideas about choosing seed sites consider a Google search for “seed websites” or any specific topics of interest to yourself.

 

If you are interested in internet crawling and HTML scraping you should enjoy working with this app. It can be tedious until you become familiar with how to Select | Copy | Paste on your device, how to use technical parts of the app like the stack, until you accommodate yourself to the crawling - this is not a road race! and until you discover which websites are “good seeds” for your particular interests - preferably those with many offsite links.

 

A note concerning Ethical Web Scraping…

The web crawler should respect the rules set by robots.txt and avoid frequent visits to any single website. MetaQuest gives you the tools to work this way. Web scraping is just like any tool in the world - you can use it for good stuff and you can use it for bad stuff. That web scraping itself is not illegal doesn’t mean you can scrape any site you want. Some sites explicitly block any sort of automated data extraction either via the robots.txt file or their Terms of Service page. You can find any terms of service page with a browser and Google search. MetaQuest gives you the tools to download and study the robots.txt file, so you can choose to visit or not visit individual sites, and to scrape or not scrape various folders and files, as appropriate.

 

Crawling and Scraping the Deeper Web!

With Meta Quest you can collect URLs of pages where you may want to extract the HTML code and data in a special data structure called the Stack. With Deeper Crawling the idea is to search any web page for links, especially for links to other web pages or web sites in other countries. Then explore those pages and sites for further links to other websites, to yet other countries, to wherever. Then continue, deeper and deeper, through the World Wide Web, throughout the MetaVerse.

 

From the opening view MetaQuest has practical, introductory lessons to get you started. Plus you will find that you can exit to any other app such as Google Maps, Google Search, a text editor and to your favorite browser, then return to MetaQuest while keeping your “breadcrumbs” intact in the Stack, so you can go wherever there is a place to go and explore whatever is to be found there.

 

Be A Member of the Open Testing Group

There are a few prerequisites. First, Google insists that you have a gmail account to join a test group for a Google Play Store app. If you do not yet have a gmail account click this link and follow the steps…

https://support.google.com/mail/answer/56256?hl=en

 

Second, to join the test group send an email using your gmail account to MultiMIPS@gmail.com. Just mention MetaQuest Open Test Group in the title or body of the email. You may add in some bits about yourself but that is not necessary. I will ASAP acknowledge that I received the email and follow with weekly or daily updates. I am hoping to have a larger number of testers so please check this out if you may be interested.

 

Third, before you run the app you will want to get a free, private API_KEY from https://ipgeolocation.io/  This will allow your copy of MetaQuest to retrieve from the internet the geoLocation that corresponds to any IP address that it finds. An illustrated set of steps to follow to get your free, personal API_KEY is here…

https://mickwebsite.com/MMWebSite/ipgeolocation.html

 

Finally, a  preview is available right here, right now…

This introductory Quest begins with an overview of the MetaQuest menu options so you gain an understanding of the app structure and flow. It then starts a quest at https://www.example.com in Brasilia, Distrito Federal, Brazil and tours throughout the internet to Durdevac, Croatia. Afterwards, you should perhaps plan on joining the Open Test Group and continue this tour through Ireland, Germany, Sweden, Turkey, and, well, to wherever… to see what you can see. Click here to get started…

https://mickwebsite.com/MMWebSite/IntroductoryQuest.html

 

The Open Testing will be followed by MetaQuest finally being released to production later this Fall.

 

Mick

MultiMIPS@gmail.com