Showing posts with label docker. Show all posts
Showing posts with label docker. Show all posts

Fix Docker Error: Permission Denied to Docker Daemon Socket (Without Sudo)

 


You just installed Docker, you’re excited to run your first container, and you type docker ps. Instead of a list of containers, you get this:

docker: Got permission denied while trying to connect to the Docker
daemon socket at unix:///var/run/docker.sock

Why is this happening?

By default, the Docker daemon binds to a Unix socket which is owned by the root user. Because you are logged in as a regular user, you don't have the permissions to access that socket.

Most people "fix" this by typing sudo before every command, but that is dangerous and bad practice. Here is the professional way to fix it once and for all.

Step 1: Create the Docker Group

In most modern installations, this group already exists, but let’s make sure:

sudo groupadd docker

Step 2: Add your User to the Group

This command adds your current logged-in user to the docker group so you have the necessary permissions.

sudo usermod -aG docker $USER

Step 3: Activate the Changes

Important: Your terminal doesn't know you've joined a new group yet. You can either log out and log back in, or run this command to refresh your group memberships immediately:

newgrp docker

Step 4: Verify the Fix

Now, try running a command without sudo:

docker run hello-world

If you see the "Hello from Docker!" message, you’ve successfully fixed the permission issue!

Note:

Adding a user to the "docker" group is equivalent to giving them root privileges. Only add users you trust. If you are in a highly secure environment, consider using Rootless Docker, which allows you to run containers without needing root access at all.

How to Fix: "SSL Certificate Problem: Self-Signed Certificate" in Git & Docker



This is one of the most common "Security vs. Productivity" errors. You’re trying to pull a private image or clone a repo, and your system blocks you because it doesn't trust the security certificate.

The Error: fatal: unable to access 'https://github.com/repo.git/': SSL certificate problem: self signed certificate in certificate chain


Why is this happening?

Your company or home network is likely using a "Self-Signed" SSL certificate for security monitoring. Git and Docker are designed to be secure by default, so they block these connections because they can't verify the "Chain of Trust."

❌ The "Bad" Way (Don't do this in Production!)

You will see people online telling you to just disable SSL verification:

git config --global http.sslVerify false

Why avoid this? This turns off security entirely, making you vulnerable to "Man-in-the-Middle" attacks. It's okay for a 2-minute test, but never leave it this way.

✅ The "Secure" Fix (The DevSecOps Way)

Instead of turning security off, tell your system to trust your specific certificate.

1. Download the Certificate

Export the .crt file from your browser (click the lock icon next to the URL) or get it from your IT department.

2. Update Git to use the Certificate

Point Git to your certificate file:

git config --global http.sslcainfo /path/to/your/certificate.crt

3. Update Docker (on Linux)

If Docker is failing, move the certificate to the trusted folder:

sudo mkdir -p /usr/local/share/ca-certificates/
sudo cp my-cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

Pro Tip: Use a Secret Scanner

While you're fixing security errors, make sure you aren't accidentally pushing passwords into your code! Tools like TruffleHog or git-secrets can scan your repo and stop you before you commit a major security leak.


Fixing Docker Error: "conflict: unable to remove repository reference"



Have you ever tried to clean up your local machine by deleting old Docker images, only to be met with this frustrating message?

Error response from daemon: conflict: unable to remove repository reference

"my-image" (must force) - container <ID> is using its referenced image <ID> 

This error happens because Docker is protective. It won't let you delete an image if there is a container—even a stopped one—that was created from it.

Step 1: Identify the "Zombie" Containers

The error message usually gives you a container ID. You can see all containers (running and stopped) that are blocking your deletion by running:

docker ps -a

Look for any container that is using the image you are trying to delete.

Step 2: Remove the Container First

Before you can delete the image, you must remove the container. If the container is still running, you’ll need to stop it first:

# Stop the container

docker stop <container_id>


# Remove the container 

docker rm <container_id>

Step 3: Delete the Image

Now that the dependency is gone, you can safely remove the image:

docker rmi <image_name_or_id>

The "Shortcut" (Force Delete)

If you don't care about the containers and just want the image gone immediately, you can use the -f (force) flag.

Warning: This will leave "dangling" containers that no longer have a valid image reference.

docker rmi -f <image_id>

Pro Tip: The Bulk Cleanup

If your machine is cluttered with dozens of these conflicts, don't fix them one by one. Use the prune command to safely remove all stopped containers and unused images in one go:

docker system prune

(Add the -a flag if you also want to remove unused images, not just "dangling" ones.)


How to dockerize your python application in docker

Dockerize your python application:






Docker is a technology which lets you build, deploy and run your applications. Docker enables you separate your infrastructure from your application. With Docker all you need to do is just write your code,.
dockerize it and distribute it in form of image. That way any one can use your application who is running the Docker.

What do you mean by Dockerize application?

Dockerize mean you write your code on your system then you prepare the image and distribute it over the internet or on DockerHub. You don't have to worry about the underlying infrastructure and dependencies.

Let's write a python program which will count the occurrence of words from a given string.


#Input : string = "Docker is a technology which
# lets you build, deploy and run your applications.";
#Count occurence of words from a given string example


def findFreq(s):
dictt = {}
strng = s.split(" ")
strr1 = set(strng)
for word in strr1:
dictt[word] = s.count(word)
return dictt
if __name__ == "__main__":
x = input("Enter your string:")
#raw_input in python 2.x and input() in python 3.x
print(findFreq(x))

#Output: {'a': 4, 'and': 1, 'run': 1, '': 80,
# 'deploy': 1, 'technology': 1, 'is': 1,
# 'you': 2, 'lets': 1, 'applications.': 1,
# 'which': 1, 'build,': 1, 'Docker': 1, 'your': 1}

Save this file with findfrequency.py in same directory. I am saving it in current directory for my convenience but you can save it anywhere and pass the absolute path.


Now lets create a Dockerfile.


FROM python:3

We need to use python in docker so we are using FROM keyword so this will create layer from python image. Means your image is based on python image. 

Now we need to run our python file so we need to add this file to Dockerfile.

ADD findfrequency.py /

Use CMD to execute commands when image loads

CMD ["python", "./findfrequency.py"]

Combine all above lines and create a Dockerfile.

FROM python:3
ADD findfrequency.py /
CMD ["python", "./findfrequency.py"]

So we have created a Dockerfile now. I saved it with the name "Dockerfile" in current directory. When you run docker build .     command then docker looks for Dockerfile if ithis file doesn't exist or file name is wrong or extension is wrong you'll get file not exists error.

Now we are ready to build image from the dockerfile. 

Open the terminal and run the below command and make sure you are in the same directory where you saved your Dockerfile as well as python file.

docker build -t myapp .


-t : This is tagging a name to your image. In this case I gave my image a name "myapp"
.(dot) : Is current directory

Ok so you have successfully build your image. Now Let's check what's inside the image by inspecting it.

docker inspect myapp

[

    {

        "Id": "sha256:c4595feabbd0b9aba4ae67037ea3c43a8c0aaf2abe6f6fd28d25b22a7cf9",

        "RepoTags": [

            "myapp:latest"

        ],

        "RepoDigests": [],

        "Parent": "",

        "Comment": "buildkit.dockerfile.v0",

        "Created": "2021-10-01T08:42:53.450488763Z",

        "Container": "",

        "ContainerConfig": {

            "Hostname": "",

            "Domainname": "",

            "User": "",

            "AttachStdin": false,

            "AttachStdout": false,

            "AttachStderr": false,

            "Tty": false,

            "OpenStdin": false,

            "StdinOnce": false,

            "Env": null,

            "Cmd": null,

            "Image": "",

            "Volumes": null,

            "WorkingDir": "",

            "Entrypoint": null,

            "OnBuild": null,

            "Labels": null

        },

        "DockerVersion": "",

        "Author": "",

        "Config": {

            "Hostname": "",

            "Domainname": "",

            "User": "",

            "AttachStdin": false,

            "AttachStdout": false,

            "AttachStderr": false,

            "Tty": false,

            "OpenStdin": false,

            "StdinOnce": false,

            "Env": [

                

                "LANG=C.UTF-8",

                "PYTHON_VERSION=3.9.7",

                "PYTHON_PIP_VERSION=21.2.4",

                "PYTHON_SETUPTOOLS_VERSION=57.5.0",

                "PYTHON_GET_PIP_SHA256=fa6f3fb93cce234cd4e8dd2be9c247653b52855a48dd44e6b21ff28b"

            ],

            "Cmd": [

                "python",

                "./findfrequency.py"

            ],


You'll see output something like above. Our python function is there inside the output under CMD tag.

Let's run the image.

docker run -it myapp   

Enter your string: This is my test to test dockerfile. 

{'': 37, 'is': 2, 'dockerfile.': 1, 'to': 1, 'my': 1, 'test': 2, 'This': 1}


See the output above and pass the desired string to count the words.

So we have successfully dockerized our application. You can send this image to others so that they can use your program and they don't have to worry about installing any dependencies which can cause your program to crash. 

How to create a Dockerfile to automate the build image process

 Creating a Dockerfile




Dockerfile is a text file which contains set of commands which are used by user to build an image via command line. Dockerfile is an automated way to build an image with set of commands.

Docker build is a command which builds an image from Dockerfile. Let's create a docker file and put some commands inside it. 

touch Dockerfile


Above touch command is used to create a new file on mac/linux. Make sure you create the docker file in current directory. Although you can create the docker file anywhere but for convenience I am creating it on current directory. So that I don't have to type the absolute path for docker file.

Always remember these three commands while creating a new docker file - 

FROM : This creates a layer from docker image
RUN : builds your application
MAINTAINER : Best practice to put the name of maintainer of the image
CMD : Run the commands inside container 

I have created the dockerfile and put the below text inside dockerfile.

FROM ubuntu


MAINTAINER DATATIPSS


RUN apt-get update


CMD ["echo","This is my first image using dockerfile"]


I am using ubuntu as base image and giving maintainer as datatipss. You can give your name. 

Now run the docker file using docker build -t myapp .  command

Here -t : tag your image or giving a name to your image and .(dot) is running dockerfile in current directory.

docker build -t myapp .

[+] Building 79.3s (6/6) FINISHED                                                                                                                             

 => [internal] load build definition from Dockerfile                                                                                                     0.0s

 => => transferring dockerfile: 152B                                                                                                                     0.0s

 => [internal] load .dockerignore                                                                                                                        0.0s

 => => transferring context: 2B                                                                                                                          0.0s

 => [internal] load metadata for docker.io/library/ubuntu:latest                                                                                         7.9s

 => [1/2] FROM docker.io/library/ubuntu@sha256:9d6a8699fb5c9c39cf71bd6219f0400981c570894cd8cbea30d3424a31f                                          7.3s

 => => resolve docker.io/library/ubuntu@sha256: 9d6a8699fb5c9c39cf71bd6219f0400981c570894cd8cbea30d3424a31f                                          0.0s

 => => sha256: 9d6a8699fb5c9c39cf71bd6219f0400981c570894cd8cbea30d3424a31f 1.42kB / 1.42kB                                                           0.0s

 => => sha256: 9d6a8699fb5c9c39cf71bd6219f0400981c570894cd8cbea30d3424a31f 529B / 529B                                                               0.0s

 => => sha256:54ab604fab8d1b3d1c8e02509cc7031f8541428051401f4122619e5968e16 1.48kB / 1.48kB                                                           0.0s

 => => sha256:ab2d02b1ec420fdb84c9f52abda403b6a0f5de904a2ecda5ae4f5cd6e4d46 27.17MB / 27.17MB                                                         6.6s

 => => extracting sha256:ab2d02b1ec420fdb84c9f52abda403b6ae0f5de904a2ecda5ae4f5cd6e4d46                                                                0.7s

 => [2/2] RUN apt-get update                                                                                                                            63.9s

 => exporting to image                                                                                                                                   0.0s

 => => exporting layers                                                                                                                                  0.0s

 => => writing image sha256:76ff7254c9ca346bfe34c42c3f7c17ad0958e70210f01f9e200cbdadb2e4f   



Ok now you have successfully created your image. Let's check and run the image using docker run imageid_or_image_name command.

Check images first:

docker images

REPOSITORY        TAG       IMAGE ID       CREATED         SIZE

myapp             latest    76ff7254c9ca   3 minutes ago   92.2MB


Now run the image:

docker run myapp

This is my first image using dockerfile



So we have successfully created our first image and ran it. 

Also you can inspect the image and get insights of it. If you are using image build by someone else so before running it you can always inspect it to check if it is legitimate image. You can inspect any image and check the author and what commands are inside the image.

In this case you can check the author is datatipss and under cmd tag it's echo command only.

docker image inspect myapp

[

    {

        "Id": "sha256:76ff7254c9ca346bfe34c42c3f7c1958e70210f01f9e200cbdadb2e4f",

        "RepoTags": [

            "myapp:latest"

        ],

        "RepoDigests": [],

        "Parent": "",

        "Comment": "buildkit.dockerfile.v0",

        "Created": "2021-09-28T17:40:34.86456821Z",

        "Container": "",

        "ContainerConfig": {

            "Hostname": "",

            "Domainname": "",

            "User": "",

            "AttachStdin": false,

            "AttachStdout": false,

            "AttachStderr": false,

            "Tty": false,

            "OpenStdin": false,

            "StdinOnce": false,

            "Env": null,

            "Cmd": null,

            "Image": "",

            "Volumes": null,

            "WorkingDir": "",

            "Entrypoint": null,

            "OnBuild": null,

            "Labels": null

        },

        "DockerVersion": "",

        "Author": "DATATIPSS",

        "Config": {

            "Hostname": "",

            "Domainname": "",

            "User": "",

            "AttachStdin": false,

            "AttachStdout": false,

            "AttachStderr": false,

            "Tty": false,

            "OpenStdin": false,

            "StdinOnce": false,

            "Env": [

                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

            ],

            "Cmd": [

                "echo",

                "This is my first image using dockerfile"

            ],

            "ArgsEscaped": true,

            "Image": "",

            "Volumes": null,

            "WorkingDir": "",

            "Entrypoint": null,

            "OnBuild": null,

            "Labels": null

        },

        "Architecture": "arm64",

        "Variant": "v8",

        "Os": "linux",

        "Size": 92173773,

        "VirtualSize": 92173773,

        "GraphDriver": {

            "Data": {

                "LowerDir": "/var/lib/docker/overlay2/76ff7254c9ca346bfe34c42c3f7c1958e70210f01f9e200cbdadb2e4f/diff",

                "MergedDir": "/var/lib/docker/overlay2/c7dp26x65obhv96p76ff9lbo1/merged",

                "UpperDir": "/var/lib/docker/overlay2/c7dp26x65obhv96p76ff9lbo1/diff",

                "WorkDir": "/var/lib/docker/overlay2/c7dp26x65obhv96p76ff9lbo1/work"

            },

            "Name": "overlay2"

        },

        "RootFS": {

            "Type": "layers",

            "Layers": [

                "sha256: 76ff7254c9ca346bfe34c42c3f7c1958e70210f01f9e200cbdadb2e4f",

                "sha256:ecd02c9154cf7dc6fa2257f9378687064ebbb87524aa7191a683650b0c"

            ]

        },

        "Metadata": {

            "LastTagTime": "2021-09-28T17:43:44.89023209Z"

        }

    }

]


That way you can check what's going to run from the image. Once you build your image you can make it publicly available for everyone. You can upload it on DockerHub and anyone can use it. 

You can build multiple images of multiple applications and distribute it over the internet so that anyone can use your docker image. 

Terraform for Data Engineers: How to Automate Your Database Setup

  Stop Manual Setup: Deploy a PostgreSQL Database with Terraform If you are still manually creating databases in the AWS or Azure console, y...