Protecting Our Searches - Setting up SearXNG
August 24th, 2022

With the use of social media to follow topics of interest, personal information is collected and used to keep us engaged. We should be able to be informed about any subject without impacting our online experience.

An experiment was performed by searching about Cambodian politics over a weekend. In no time, there were notifications on Twitter about a Cambodian politician running for office. Multiple users searched for “2020 elections”. There were different results.

After getting the same type of information over multiple platforms, it became clear how easy it is to become biased on a subject.

Search engines create a profile base on searches and the information it feeds back is tailored to that profile, perhaps, providing unpopular, biased, and incorrect information.

Personal information is sold to third parties.

There is concern about how data is collected and who is entrusted to have access to it.

There is no telling how it will be used in the future and its effects on users.

It is not a matter of trusting search engines. The US government has been collecting personal data through tech giants for over a decade as the Guardian reported on the NSA’s collection of data from Yahoo, Apple, Google, Microsoft, Facebook, and AOL.

There have been efforts around the world to take action in privacy protection.

While searching on the internet in incognito mode can give a false sense of privacy, this removes existing cookies from future browsing and does not record browsing history in the browser. The IP address and browser fingerprint are exposed to search engines.

DuckDuckGo may not be as private as thought after a security researcher found a "privacy hole", a confidential search agreement in which the DuckDuckGo browser doesn’t block all Microsoft trackers.

Screenshot - privacy engineer comment on DuckDuckGo's CEO Twits
Screenshot - privacy engineer comment on DuckDuckGo's CEO Twits

A way to protect your privacy is by taking matters into your own hands not leaving it to chance that search engines are acting with the best intentions. By saying taking the matter into your own hands, I'm talking about SearXNG.

What is SearXNG

SearXNG is a fork of Searx. It is an open source "metasearch engine", meaning it doesn't itself index the internet, but instead sends search queries out to several other search engines*.* It then brings those results together, presenting them in a unified search result.

How does SearXNG help protect our privacy?

  • SearXNG removes private data from requests going to search services.
  • It doesn’t forward anything from third-party services through search services like ads.
  • It removes private data from requests going to search engines by not sending cookies.
  • It generates a random browser profile on each request.

Using a hosted SearXNG

I have hosted an instance of SearXNG in the cloud which you can use at https://leidy.info. In the next part of this article, I will show how to deploy an instance of SearchXNG for free.

Hosting SearXNG in the cloud

The quickest and easiest way is, to host SearXNG in the cloud is by using the searXNG docker-compose image. I chose to use AWS because I think is very easy and besides, they provide a 1-year free tier. However, you can do it using any server of choice.

Creating an EC2 Instance

  1. Create an AWS account and head over to the EC2 console.
  2. Click on “Launch Instance”.
  3. Select Ubuntu version 20.04 LTS.
  4. For instance type select t2.micro “Free tier eligible”.
AWS Instance type dashboard screenshot
AWS Instance type dashboard screenshot

Create a new key pair

To securely connect over SSH to the server. Save the pem file in a secure location and set its permissions to read only by you, or chmod 400 if using Mac or Linux.

AWS Key pair dashboard screenshot
AWS Key pair dashboard screenshot

For the security groups

Set ssh to allow your IP address (if you do not have a static IP you may have to come back and update this if your IP address changes).  Add HTTP and HTTPS from anywhere so anyone can connect via their browser and use SearXNG.

AWS Security Groups dashboard screenshot
AWS Security Groups dashboard screenshot

Finally, click on “Launch Instance” and you now have an Ubuntu server.

Create an elastic IP

Elastic IP addresses never change and you can assign them to any resource.

  1. Under the dashboard, select “Elastic IPs”.

  2. Select “Allocate Elastic IP Address” then

  3. “Allocate”. The Elastic IP address will get created and will show on the list.

    Now we need to associate this IP address with our EC2 Instance.

  4. Select the created Elastic IP address.

  5. Click on “Actions”.

  6. Then click “Associate Elastic IP address”.

    You can view the details of the created instance including the newly added elastic IP by going to the EC2 instance dashboard.

AWS Elastic IP address dashboard screenshot
AWS Elastic IP address dashboard screenshot

Connecting to Instance

SSH to your instance:

ssh -i /path/to/your/private_key_name.pem ubuntu@your.instance.ip

Installing docker and SearXNG

  1. First, make sure the server is up to date
    $ sudo apt update && sudo apt upgrade -y

  2. Install docker
    $ sudo apt install docker.io -y
    $ sudo apt install docker-compose -y

  3. Change to the /usr/local/ directory
    $ cd /usr/local/

  4. Clone the searxng docker container
    $ git clone https://github.com/searxng/searxng.git

  5. Change the directory to the one you just cloned

    $ cd searxng-docker

  6. Use sed to replace the string in the settings.yml file secret_key: "ultrasecretkey with a random 32-digit string using the OpenSSL command $ sed -i "s|ultrasecretkey!|$(openssl rand -hex 32)|g" searxng/settings.yml

  7. Edit the .env file and insert your domain name (example.com) after the SEARX_HOSTNAME= variable and add a valid email to the end of the LETSENCRYPT_EMAIL= variable. This is required for getting a valid Let’s Encrypt certificate.

  8. Run SearXNG
    $ docker-compose up

docker-compose is telling docker how to build our SearXNG environment. There is also an image using Caddy, which is a reverse proxy server with bundled support for getting SSL certificates from Let’s Encrypt. It is going to build 3 docker containers and connect them and make it work like magic and all you have to do is type one command.

$ docker-compose up -d

You can personalize your hosted SearXNG by going into searxng and updating the settings.yml file

For example, here I set it up to autocomplete using duckduckgo

settings.yml file
settings.yml file

Pointing your domain to your instance

You will need to point your domain to your EC2 instance public IP address where SearXNG is up and running. To do this, head over to your domain provider where you bought the domain and go to the DNS section to add records. Create an “A Record”, the host will be at the root or current origin “@“ and the value will be your EC2 instance IP address (the elastic IP address you created previously).

Final thoughts

You can use a public hosted instance of SearXNG (hosted by another company or person) however, you don’t know what is happening in the background with those requests.

The best thing you can do is to host your instance as explained above to have full control and know what is happening and not happening when you use it.

If you need additional protection, you should be using a VPN to hide your IP address.

It is your responsibility to protect your privacy even if you are not browsing the internet with malicious intentions. Remember, no one can guarantee you that in the future all the data collected about you won’t be used against you.

Deployed SearXNG on the server
Deployed SearXNG on the server
Subscribe to Zoist
Receive the latest updates directly to your inbox.
Nft graphic
Mint this entry as an NFT to add it to your collection.
Verification
This entry has been permanently stored onchain and signed by its creator.
More from Zoist

Skeleton

Skeleton

Skeleton