Getting DNS working with an Ubuntu 20.04 virtual machine installed on an Ubuntu 19.10 host

While taking a look at the next LTS release of Ubuntu - I found after I had spun up the new image in KVM on an Ubuntu 19.10 host that the DNS would not resolve - which scuppered me taking a really good look at it.

Now, they have moved DNS resolution into systemd for a while now, and on the host machine this has not caused me an issue. I have to say though, it appears to me that using systemd to resolve DNS is not only overly complicated but a waste of everyone's time - but I'm sure someone must appreciate the value of it.

I tested that the network was working correctly and the virtual machine could access the DNS resolver if it had been configured correctly with the following command:

dig @192.168.1.1 www.ubuntu.com

This worked, so I knew that the virt. machine would work if the DNS resolver was working correctly.

So, how to fix the issue. I tried several methods - each trying not to disable the systemd service - but all met with failure with my testing , so in the end I decided to just turn it off and use the tried and trusted /etc/resolv.conf

The commands to achieve this are:

systemctl stop systemd.resolved
systemctl disable systemd.resolved

Then edit the following line in the file 

/etc/NetworkManager/NetworkManager.conf 

[main]  

dns=default

Then remove the link in /etc

rm resolv.conf

Now create a new resolv.conf in /etc with the name of the nameserver you wish to use i.e.

namesever 192.168.1.1 < or whatever yours is >

The you need to restart the NetworkManager 

sudo systemctl restart NetworkManager

This worked perfectly and the virtual machine is now happily resolving DNS correctly.

None of the above is destructive and can be reversed if the systemd could be made to work, but as this was only a test machine, I decided I had wasted enough time on it.

NB. While working on some more virtual machines I came across this blog post which offers a more elegant solution to this DNS problem for Ubuntu/Debian based distros.It allows you to keep DNS resoltion in systemd - so I can't have been the only person having issues with it.

Solve local DNS issues in Ubuntu and Debian

 

How to use pi-hole with a Docker container on your Mac Laptop to stop unwanted internet adverts.

If you are fed up with pointless internet advertising on sites you visit, here is a great additional service you can install on your local machine - or more importantly for your network to stop it dead. I shall not go through what this product is as here is a link - Pi-Hole.

Basically, you need to install the Docker application on you laptop or desktop so that running up the pi-hole docker container is straight forward. You can get docker for Mac here. For the network installation, a Linux server virtual machine or docker container on a machine continuously running would make sense.

Then you need to clone the pi-hole docker git repository to your local machine

https://github.com/pi-hole/docker-pi-hole.git

Change into that directory and run docker_run.sh 

Once the script has run - it will spit out an admin password that you will need to remember to log into the web-based admin screen.

You can look at that by pointing a browser tab at http://127.0.0.1

Once logged in you will see something like this.

 The last part of getting this working on your laptop is to point the DNS resolution of OSX to point to the localhost - as pi-hole is now listening on port 53. Again for network-based installation you would point this at the IP address of your server running the service. You can also then setup that IP address in your routers DHCP settings so any machine on your LAN will get the same protection as they will push all their traffic through the new DNS server.

You can then run a test from the command line to make sure all your DNS requests are going via your new DNS service like so:

dig www.ubuntu.com

;; Query time: 23 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 13 11:16:40 GMT 2019
;; MSG SIZE  rcvd: 139

You will notice the response comes from your machines localhost - so all is working. With the settings of pi-hole you can specify several upstream DNS resolvers which also keeps your DNS queries out of the clutches of Google. There are many options - but I tend to use OpenDNS and 1.1.1.1

Enjoy. 

 

Staying safe on the Web - what can I do to make my browsing more secure and leach less data?

Friends and family often ask me about technologies they can use to make their lives just that little bit safer on the Web. So to save me having to answer the same questions repeatedly - I thought I would write a blog post to just highlight the tools,apps and extensions I use to make it better than just connecting to the web and hoping for the best.

• Use a VPN whenever and wherever you are. There are so many good and inexpensive examples to use these days - there is really no excuse not too. I hear good things about these ones Mozilla VPN or Proton VPN. Make sure when choosing a VPN that the provider guarantees to not keep logs - very important. I use my own - but those should be a good ones. This will work with you PC, iPad, mobile phone. So you will be covered whenever you decide to get some free internet in a cafe  you have never been to before :-)
• Use Mozilla Firefox, Safari or Opera Browser as your main browser. I know Google Chrome offers many features, but can you honestly trust Google to not be constantly looking to take your data and use it? I certainly don't trust Chrome anymore.
• Install a good set of extensions to stop trackers and unwanted information leaching.
• Adblock Plus
• Ghostery
• uBlock Origin or uBlock Origin Lite (needed for Chrome now)
• DuckDuckGo privacy essentials
• Privacy Badger
• HTTPS Everywhere

• Don't use Google as your default search engine - switch to using DuckDuckGo or Qwant- they are an option on all modern browsers - just change the default. You will be amazed at how all the targeted ads suddenly stop appearing everywhere - because you will have stopped Google building a complete profile of you on the web.
• Never use Facebook or Twitter to log into another site - always select to create a new account with your email and a strong password.
• Use a password manager to ensure strong passwords on all sites you use. Three good examples are Bitwarden, Lastpass,1Password or KeepassXC. 
• Where sites allow it use 2FA - 2 Factor Authentication - on all sites. Not all sites do - but check and where you can implement it. There are many apps you can use,  

1. Bitwarden Authenticator,  
2. FreeOTP, 
3. YubiKey Authenticator, 
4. Protectimus Smart OTP, 
5. Lastpass Authenticator, 
6. Google Authenticator.

• Turn on Firefox browsers "DNS over HTTPS" to keep your ISP from snooping on your site visits - it's to be found in Settings/Privacy&Security/DNS of HTTPS - might have to scroll down a little to find it.
• If you must use Facebook - I recommend you don't - then install an extension that puts it in a sandbox container - Firefox FB Container. This will reduce the amount of data you will leach from that app. 
• Talking about Firefox containers - using these you can isolate all sorts of other apps as well , using this add-on.
  Firefox containers - Howto
  
• Also for facebook - look at this article and turn it off - How to delete Facebooks off page tracking of you web surfing. There are many other settings, turn as much off as you can.
• Always look to use an anti-virus products on your phones, tablets and computers - there are many to choose from - I use AVG.
•  Make regular backups - so if you machine does get hijacked you have always got access to you valuable files. This can be to a secure cloud storage area as well as local USB type storage disk. Do this weekly, and occasionally attempt a restore to some files to make sure your backups are good.
  
• Glasswire is a great tool for keeping an eye on your Windows and Android machines activity and is worth installing https://www.glasswire.com/. They promise a Mac version soon.
• Useful YouTube video on securing Google Chrome if you must use it https://youtu.be/9lIMSzrjUrU
• Switch on DuckDuckGo anti-tracking feature on your Android phone. duckduckgo-app-tracking-tool-beta-android-users
• With Spam a constant threat there are now useful email relays/proxies which hide your real email address and offer security and spam prevention for free. There are two good ones Mozilla Relay or use DuckDuckgo email service. Worth the time and effort to setup and use.
• Switch to using as many open source tools as possible to stop being held captured by the technology Oligarch's. A good example is use LibreOffice instead of Microsoft office. Europe is doing this and moving away from these tools, so now is a good time to take that step. 
• Use the DuckDuckGo Extension to Block FLoC, Google’s New Tracking Method in Chrome Stop FLOC 
   
  Specifics for AI
   
  With the rise of AI tools everywhere, it makes sense to know if what you are reading is AI generated or not, this add on for Firefox helps with that task.
  AI Tool checker
   
  Also with AI tools now being present in your browser of choice, you need to make decisions on how you use that additional functionality. Picking the right model to use is the first choice, I tend to use Mistral, as it is European based. You need to understand how it works on your system, what data gets sent where, and how you can control that. 
   
  It is also worth being aware that as AI models require more and more data, that your personal information and posts on the internet are going to be consumed. This has many problems, especially with images that can then be manipulated with AI video to produce all sorts of unsavoury output. The need for digital privacy has never been higher and the thought before one posts is now a necessity.
  Linked-In is about to start mining everyone's data on their site, so make sure you go into your settings and turn that ability off. Also check all of the apps you use , and turn off all that data mining where you can.
   
  If you want to use AI tools safely and local only to your machine, then follow the advice in this article I wrote earlier in 2025.
  Truth about AI in Spring 2025 
  
  
   
  

There are other add-ons to stop javascript - which can stop a lot of nasty attacks - however - it can make a huge difference to the way the web looks and feels and a lot of sites depend on it. So unless you know what you are doing , I would stay clear of that to start.

I  also use pi-hole, a network-wide tool to stop unwanted advertising in its tracks. I will be writing an article shortly on how to set this up on your local laptop or network server.

Here is the article on using a docker container to run pi-hole - but you can use a virtual machine or a dedicated little machine like a Raspbery-pi to achieve the same thing. It is incredible useful and effective to stop all sorts of rubbish coming into your machine and network.

Setting up Pi-hole locally using Docker

Updated: 15/7/2019 
Updated: 13/11/2019
Updated: 9/02/2020
Updated: 1/3/2020
Updated: 16/6/2020
Updated: 11/09/2020
Updated: 15/03/2021
Updated: 14/05/2021
Updated: 8/8/2022
Updated:4/12/2022

Updated:8/2/2025

Updated:20/5/2025 

Updated:24/8/2025

Update:22/10/2025 

Updated:7/6/2026 

Useful reading on the Topic :

• Electronic Frontier Foundation
• Security Crash course

DNS resolution in Docker containers with Ubuntu Artful on AWS

This post is solution to a problem I discovered - so I hope others will find it useful.

Spinning up AWS Ubuntu Zesty - 17.04 - images with Docker installed was straight forward with Ansible and Terraform , but then arrived Ubuntu Artful - 17.10 , and the containers spun up could not resolve DNS, regardless of which version of Docker I installed.

After a lot of testing , it appeared to me that the host computer was passing through the wrong DNS server entry into resolv.conf within the container - so it would never work.

The Solution:

With systemd and docker, the preferred way to change a daemon setting is to create a new file in /etc/docker called daemon.json.

In that file add the following to get it use the AWS VPC default DNS resolver - 10.0.0.2 - like so

{
   "dns": ["10.0.0.2"]
}

Restart the docker daemon , and the containers can now resolve DNS. There may be other ways to resolve this issue, but this works perfectly , and uses methods preferred by the docker community.

I hope this helps others who may run into this problem.

Other settings that can be made in that file can be found here. Dockerd settings documentation

Getting apps to work in cutting edge Ubuntu Docker containers when they grizzle about locales.

Whilst running up an Artful Aardvark Ubuntu docker container on a Zesty Ubuntu host server, I received a message that it couldn't load certain apps due to the locales for UTF-8 were not configured.

This was annoying, but I initially worked around it , by installing the locales-all package into the container - it worked - but it bloated the container considerable.

There is a better and simpler way, which I found after a lot of digging around in the Docker documentation, as others must have hit this issue before.

What you need to do is set the following env variable in your Dockerfile when you build your container, and the problem does get solved for most apps, like tmux and screen within the container.

ENV LANG C.UTF-8

If you find that this doesn't cure it for your application , you may need to move to the next step and include the following in your Dockerfile.

       
RUN apt-get update && apt-get install -y locales \
&& rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A \
/usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8
       
 

Getting an elastic search shard to relocate after a cluster nodes disk had become full

I came into work today to find one of the test environments elastic search(ES) nodes had run out of disk space. A bad job had gone berserk overnight and filled the logs - which then filled ES nodes disks.

So what to do with a volume with 100% utilisation? After chatting to a few colleagues , it was decided to delete one of the earlier indices - as the data in the dev environment was not life or death.

I first tried this from the GUI - which didn't work at all , so I switched to the CLI and issued the following

curl -XDELETE curl localhost:9200/mydodgylogs-2017.05.08 

That did the trick and we now had 80% disk utilisation, so I was expecting the shards to sort themselves calmly out. Unfortunately no go - there was one shard that was still refusing to relocate and it was effectively marking the cluster as yellow, so I had another chat with my colleague and he informed of a recovery log file which could make this happen. Effectively the disk being full had left the shard in an unstable state, and needed some help to sort itself out.

So on the cli again, I found the offending shard directory in the correct index and removed the following file

 mydodgylogs-2017.05.08/4/translog/translog-1234567890.recovering

As soon as that was removed the shard relocated fine and the system went back to being happy and green.

As it took some time with a few colleagues to get to the bottom of the problem, I thought others may find it useful in the future.

Running the normal health check command then gave me the following healthy output.

curl localhost:9200/_cluster/health?pretty

{
  "cluster_name" : "myclustername",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 45,
  "active_shards" : 90,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0

}

Using S3cmd to list the ACL's of all the files in an S3 bucket

In the absence of any replies from the S3cmd forum, I managed to use a command line hack to get the ACL status of all files in a bucket with this:

 I tried parsing the whole bucket with an "astrix" as an option , but that didn't work with version 1.0.1 or 1.0.5 of S3cmd. 

s3cmd -c ~/.s3cfg_uk2 ls s3://test-hubs/ | awk '{print $4}' | sed 's/s3\:\/\/test-hubs\///g' | xargs -I file s3cmd -c ~/.s3cfg_uk2 info s3://test-hubs/file 

 This is all on one line - and the xargs option is a capitol "i" and not an "el" as it appears here ;-) If anyone can see how to refactor this to make it more efficient be my guest, but it works.

Or indeed answer my original question with a snappy command line option to s3cmd ;-)


If you want to see if there is "anyone" access to a file you will see "anon" as the ACL setting , so you can search on that if you want to look for globally available files - which is what I wanted to do.