Custom Search Engines using the Mojeek API and Mojeek Focus

17 October 2024

4 min

The web is vast and replete with both incredibly useful information and spam. Search engines help you discover and navigate across its huge surface. Sometimes you know where you want to get something from and you just search on that site, even if you may use a search engine to navigate there.

In between these two extremes of discovery and navigation, there are occasions when you want to explore a particular niche, topic, sector, or theme and so on. These themes might be very well covered across a small set of multiple sites, when compared with the vast number of sites on the web. When doing these narrower searches across the web, the big search engines and their search proxies are working against you, since they give you little agency over what is searched.

This is where Mojeek can give you back that agency, by allowing you to search across a set of sites using Focus, either on mojeek.com or via the API. Here we will show you three examples which illustrate the variety of things that can be done with these powerful features:

Search across several online encyclopedias
Search across US State government websites
Search websites, blogs and advice on digital accessibility (A11y)

Encyclopedias

Wikipedia is a goldmine of information and a marvel of what the human collective can build. What it provides in depth is recognised. But what is often forgotten is that it’s has become the “gospel” of information. Since just one page is allowed on each topic, some argue the content of each page represents an amalgm of the consensus view, and that’s good. Others will point out that dissenting voices are thus suppressed.

So what if we always searched across multiple online encyclopedias? Below is how we did that in Mojeek Focus; on the left is the Focus dashboard set-up with 11 encyclopedias, and on the right are the results for a query.

The Focus for Encyclopedias being created using the Focus dashboard on one side, next to a search for

If you want to use this Mojeek Focus, you can use the Restore function on your Focus dashboard and paste this:

{"Encyclopedias":"i=www.britannica.com,en.citizendium.org,www.conservapedia.com,www.encyclopedia.com,encycloreader.org,everything2.com,handwiki.org,rationalwiki.org,scholarpedia.org,encyclopedia2.thefreedictionary.com,en.wikipedia.org"}

N.B. double click to highlight one word and then click once more to select all content within the scroll box.

With Focus we have limited these searches to be across 25 sites so that it can be used without JavaScript. But with the API we recently made some major upgrades so you can now do searches across up to 100 sites.

US States

Suppose you wanted to find out information across the 50 USA state websites. You may think this can be done by searching across all .gov websites, using the site:gov operator in the Mojeek search box. But in this case your results are likely to be dominated by federal .gov websites. Also there is one state, Florida, which uses myflorida.com, not florida.gov.

So here we take advantage of the higher limit in the API and can easily search across all 50 domains for the states. We can do this with the site include parameter &fi, as shown in the API documentation, executed like this:

&fi=.alabama.gov,.alaska.gov,.arkansas.gov,.az.gov,.ca.gov,.colorado.gov,.ct.gov,.delaware.gov,.ehawaii.gov,.georgia.gov,.idaho.gov,.illinois.gov,.in.gov,.iowa.gov,.kansas.gov,.kentucky.gov,.la.gov,.louisiana.gov,.maine.gov,.maryland.gov,.mass.gov,.michigan.gov,.mn.gov,.mo.gov,.ms.gov,.mt.gov,.myflorida.com,.nd.gov,.nebraska.gov,.nv.gov,.nh.gov,.nj.gov,.nm.gov,.nc.gov,.ny.gov,.ohio.gov,.oregon.gov,.pa.gov,.ri.gov,.sc.gov,.sd.gov,.texas.gov,.tn.gov,.utah.gov,.vermont.gov,.virginia.gov,.wa.gov,.wisconsin.gov,.wv.gov,.wyoming.gov

Below is part of the JSON response for a search made to the Mojeek API, using the domain list above:

The JSON response for a search using the US States Focus shown above. Results have URLs, titles, descriptions, and sizes.

Accessibility

For another example of how you can use the API, we created a showcase search engine for digital accessibility, aka A11y. With the help of Joe Devon, the co-founder of GAAD (Global Accessibility Awareness Day) and Tim Dixon we put together a set of 56 sites covering projects, companies, bloggers and advocates who write about the topic. To see the list of sites we used, and even to suggest what we might add, you can use or contribute on the Focus templates Github repo.

JSON output for an example search is shown below, this time showing the details available in the head section which shows output relating to the query and keyword matches. Full details of the JSON response format can be seen in the API documentation.

The JSON response for a search using the Accessibility Focus. The head section up top has been expanded, showing some extra information such as query, nword, rankm, dups, nph, and more. Full details can be found in the link above.

Search Engine Lists

With Mojeek Focus and/or using the Web Search API you can use and build any number of bespoke search engines. The three above are examples, and if you wanted more inspiration you can see other examples of search engines created by ourselves and the Mojeek community on the Github Focus templates repo. Your ideas and contributions are most welcome there or by getting in touch.

colin