This Blog started out as a Top Tip post but then I realised there was more content than I had first imagined. I wanted to share with you how I organise my search engines for conducting my OSINT investigations.
Any OSINT practitioner knows that you should search across as many search engines as you can to find results, as they all crawl and index differently and even proxy search engines such as DuckDuckGo & Startpage can return different results than what you will find on Bing & Google respectively.
Don’t forget the use of a VPN can also alter your search results regardless of the search engine you use. Check my blog on how to use a VPN for OSINT https://www.cqcore.uk/effective-use-of-a-vpn-in-osint/
Ideally you should have in mind the search engines you want to use for your OSINT deployment. Can you use Google, or does your OPSEC require you to use Startpage. Are you investigating someone in another country, so you need to use a local search engine or Google’s country specific search engine. Check out my search engine resources, which will assist you in making your choices. https://github.com/cqcore/Search-Engines
I will start with Firefox as it is quite a streamlined process. I have shown you in my previous Telegram VM blogs how to add a new search engine to Firefox but I will repeat the process for ease.
You first need to visit the search engine home page of choice. In the below image I have navigated to Searx. Right click in the URL bar and you will be presented with a drop down menu where your search engine of choice will be at the bottom. Click, ‘Add’ and the search engine will be added to the default option in Firefox’s settings. If you do not see the, ‘Add” button it is because the search engine does not support, ‘OpenSearch’ and therefore you can’t add it.
Navigate your way to the, ‘Search’ settings in Firefox and you will find your search engine shortcut menu. As you can see from the below, I have added Yandex, Startpage, Searx and Qwant.
Navigate your way to the search bar and type in your keyword, as you do this you will be presented with your search engine options. I have highlighted the Searx icon. Firefox will then auto populate Searx for you.
The process on Chrome is not as seamless as it is on Firefox. If you navigate to the settings menu within Chrome and then select the search engine menu, you will see a list of default search engines you can choose from.
You can add further search engine to the list by following the below steps. First obtain the search engine URL you would like to add. Below I am going to add Startpage, so I need to copy the Startpage URL.
Return to the searh enigine setting menu, you can then add your chosen search engine to the list by selecting the. ‘Add’ button.
Fill out the details in the menu bar and then select add. Your search engine will be added to the list of shortcuts you can choose from in the search bar.
Open a new tab and type in the saved search engine you want to use. In the first image below I have typed in Startpage. You then need to press the space bar followed by the tab button which will then bring up the search option, shown in the second image below. This method can differ depending on the search engine. If you are struggling, type, ‘search’ after the search engine name and then your keyword.
You can then type in your keyword, as seen below.
What you will see in the URL is that Google is querying Startpage and returning the results for you.
https://www.google.com/search?q=startpage+search+bbc#ip=1
I am not particularly happy with the above method. I don’t discount this method entirely. Allowing Google to query other search engines may well have its OPSEC benefits, especially if you are using a local search engine in another country; that way you can obfuscate your searching of that local resource.
In the below example I have used Google to search the Ezilon search engine which is a search engine used in the Balkans region of Europe.
Another way to add search engines is to navigate your way to the search engine of choice and search for a keyword. Any keyword will do. Go through the same steps as previous but this time copy the full URL including keyword. Paste that URL in the URL box but replace the keyword with ‘%s’ as can be seen below.
Now go back to Chrome’s search bar and type in your search engine shortcut. In my case it is ezilon.com followed by the ‘tab’ key. This will then bring up a search bar as seen below.
As you can see I have used Tirana again and my results are returned below.
I prefer to save the search engines I intend to use as shortcuts on Google’s home page. I don’t want Google obtaining my results for me. This is probably the easiest way to search using other search engines.
To add a shortcut is straight forward. On Google’s home page select the, ‘Add shortcut’ icon. When you have done that, you will be presented with the below box where you can complete the mandatory fields. In my case I want to add Mojeek.
Once completed, selected done and your search engine of choice will be added to Google’s home page.
As you can see from the above I have added 3 search engines. Simply click which search engine you would like to use.
Another method I use to manage my search engines is to save them in a folder on my bookmark bar. These search engines are all pre-saved in a folder.
You can always open all the search engines in one go if you wanted and populate your keyword of choice.
The final method I have is to use a bespoke OSINT tool. The inspiration for this method is IntelTechniques and is based on their search OSINT tool. IntelTechnquies has build a tool for searching keywords and it is available on their site.
https://inteltechniques.com/tools/
We have already seen from how Chrome fetches our search results for the other search engines we had added to it. Below is the Startpage URL string, I showed you earlier,
The section we are interested in is the ‘search?q=’
You can build your own bespoke search engine query tool selecting the search engines that you prefer to use, once you know how each search engines query process works.
One search engine that is not included on IntelTechniquies search engine tool is Mojeek. We can find out how Mojeek URL string is constructed by using Mojeek to search for the BBC as a keyword.
https://www.mojeek.com/search?q=
As you can see it is ‘search?q=’ so you could add the above URL to the IntelTechniques OSINT search engine tool. The URL structure will not always be the same. Below is how Swisscows URL string is made up.
https://swisscows.com/en/web?query=
This is as far as I can go with showing you how to build your own tool as I do not want to infringe on IntelTechniques license. If you want to learn more I highly recommend you either buy IntelTechniques training or seek out some other appropriate training / tutorials.
My personal preference is to use Firefox and populate the search engines I want to use however I find Chrome is better for OSINT extensions, so I tend to mix and match. I also have a copy of the OSINT search tool available. You can pre-build it with the search engines you want to use for your particular OSINT deployment. Don’t stick to one browser and one search engine you could be missing out.
Copyright 2023 © cqcore All Rights Reserved