Kashish Lakhara

What is DevOps?

Kashish Lakhara — Sun, 11 Dec 2022 19:43:21 GMT

Introduction

Do you know that DevOps can accomplish tasks in minutes which use to take hours or even days of work. So the companies can work on the business or the product. And maybe that's why the IT industry is adopting DevOps at such a rapid pace.

But what exactly is DevOps?

Is it a software?
Is it a tool?
Is it a company?

So many explanations are out there:-

Some say it's all about Automation/Scripting, is it?
DevOps is about culture, is it?
It's about tools, what tools?
Is DevOps cloud computing?

To understand it first let's discuss Software Development Life Cycle (SDLC).

Software Development Life Cycle (SDLC)

Software development is a very well-defined and organized process. It consists of multiple phases.

1) Requirement gathering and analysis: In this phase, information is collected like product features, what users, how it will be used, user requirements, current market states, etc

2) Planning: What do you want? It determines the cost and resources required for the implementation of the product and also the risks associated with it.

3) Designing: In this phase, architects design the software based on the inputs from the previous phase. Architects produce design documents. These act like a roadmap for the developers.

4) Development: This is where the developers write the software code based on the design.

5) Testing (Quality assurance): In this phase, the software is tested by software testers for any defects. Software is promoted to production only after fixing all the issues.

6) Deployment: In this phase, the software is deployed to the production environment so users can start using the product. It's the responsibility of the system admin and the entire operations team to make sure the software is up and running all the time.

7) Maintenance: It is a balance between regular changes and uptime.

This entire process is called Software Development Lifecycle (SDLC).

There are different models in SDLC:

Waterfall
Agile
Spiral
Big Bang, etc

You can refer to these models as paths or roadways to reach the same destination and choose a path based on different factors like cost, risk, and time taken to reach your destination.

Now let's discuss the Waterfall model and the Agile model in brief.

Waterfall Model

In the waterfall model, development happens in step by step manner. Each phase must be completed before the next phase begins. Requirement completes, then only planning begins. When everything is completed, all the features will be tested and then the maintenance phase starts.

It is very difficult to integrate new changes given by the client or change something that didn't work well in the planning. To do so, we have to almost restart the cycle which can be very expensive and time-consuming. This will lead to delays in delivering the software.

Agile Model

In the Agile model, instead of developing all the requirements for months, it is divided into smaller lists. Work on a list for 2 to 4 weeks and then move on to the next list. In other words, the entire process of building the application is divided into small sprints. Sprints generally last for 2 weeks (The team decides the length of the sprint). After every sprint, the client can give feedback so the team can make the changes quickly and effectively. The product is delivered faster as compared to the waterfall cycle.

Here, the product is tested only on the developer's environment and not on production. It's the operations team's job to deploy the application and make sure the application is up and running. But at times, the application fails in the production servers. This may be due to misconfigurations, or dependencies that didn't match, it can be anything. The operations team has no idea what's going on so they send the application back to the development team. Due to these failures and fixes, there is a delay in delivering the application.

Dev vs Ops

Dev and Ops are poles apart. Dev is agile, all about regular and quick changes. Ops is ITIL driven, and provides a stable environment for the application.

There is a big wall of confusion between these two teams. Developers toss their code over the wall and the Ops team's responsibility is to deploy the code on the servers. Developers complain about delays in deployments. The ops team complain about unclear instructions.

Due to frequent delays and errors, the consumers are not happy. And unhappy consumers cloud mean direct business loss.

This is where DevOps comes in.

What is DevOps?

DevOps is an evolution from the Agile model. The Agile model is great for developing and testing out solutions. It addresses the gap between clients and developers.

DevOps addresses the gap between the ops team and the dev team. So we are bringing the development team and the operations team together into a single team. Now they can work smoothly because they are integrated. The wall of confusion is brought down and both teams work together.

Automation training and instructions are given to every team across the board. Automation of each and every task in the code delivery processes like code build, code testing, software testing, infra changes, deployments, and everything that comes along the way.

DevOps phases

The entire workflow in software development is divided into eight phases. This is quite similar to Agile as DevOps and agile are very closely related.

Plan:- In this phase, the client and software development team discuss the project goals and accordingly create a plan.
Code:- In this phase, the development team design and code the application based on the design. They use Version Control System (VCS) like Git to maintain the code.
Build:- In this phase, the application is built using tools like Gradle and Maven. Gradle and Maven take code from different repositories and combine them to build the complete application.
Testing:- Here, the application is tested to ensure the software quality. Automation testing tools like Selenium and JUnit are used in this phase.
Integrate:- After completing testing, new features are integrated automatically into the codebase. Tools like Jenkins are used in this phase.
Deploy:- In this phase, the application is deployed from the development server to the production server.
Operate:- Tools like Chef and Ansible are used to perform operations like configuring servers and provisioning them with the required resources.
Monitor:- In this phase, you monitor the entire environment. Monitoring allows us to identify the specific issues of specific releases and understand the impact on end-users.

Companies following DevOps:-

Netflix
Adobe
Google
Microsoft
Nvidia
Cisco
Oracle
Sony
and many more...

DevOps Advantages

Continuous integration and continuous delivery ensure faster time to market.
Collaboration between developers and the operations team is improved.
Complexity of maintaining the application is reduced.
Time to create and deliver an application is reduced.

The Evolution of Computing

Kashish Lakhara — Sat, 08 Oct 2022 06:01:38 GMT

Physical server

A physical server is wholly utilized by a single customer. You have to guess what your capacity is going to be and you're never going to 100% utilize that machine. You'll overpay for an underutilized server. You can't scale vertically, you can't resize it. You need a manual migration. Replacing a server is very difficult. You're limited by one host operating system. Running multiple apps is not a good practice for these machines because it may result in conflicts in resource sharing. Physical server provides more security, privacy, and full utility of underlying resources. If you have a virtual machine or anything above that, there is more responsibility on the cloud service provider to provide a secure machine.

Virtual Machines

You can run many virtual machines on one machine. With hypervisor you can run and manage multiple virtual machines. Here, a physical server will be shared by multiple customers. You have to pay for a fraction of a server. You'll overpay for underutilized part of the virtual machines because your app is not going to utilize 100% of memory, cpu, etc. If you run multiple apps on a single virtual machine, then it may result in conflicts in resource sharing. Its easier to export/import images for migration. Its easy to scale vertically or horizontally. Virtual machines are the most common and popular offering for compute.

Containers

Here the idea is to run multiple containers on a virtual machine. Docker daemon manages the docker objects like containers, images, volumes, etc. You can maximize the capacity because you can easily add new containers and resize those containers use up the remaining space. The containers share the same underlying OS so containers are more efficient than multiple VMs. You can multiple apps without being limited to the same OS requirements and it will not cause conflicts during resource sharing.

Docker Networking

Kashish Lakhara — Sun, 28 Aug 2022 09:20:12 GMT

Docker containers are pure magic. They are very fast, and lightweight. But...How in the world networking work? When we add these containers to a host are they on their own network? Can we create more networks? Can we isolate them? Can we expose them?
There are seven different types of networks which can be deployed with Docker containers. In this blog we will take a look at five types of network.
To follow along with this blog you need to know what docker is and a host; A linux virtual machine where we can install Docker. Here I'll be using Ubuntu 22.04 LTS as OS.

1) Default network

Now lets jump into the terminal and look at out network interfaces.

ip address show

Output for me is

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlp2s0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether cc:6b:1e:5a:7b:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.52.189/24 brd 192.168.52.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 2019sec preferred_lft 2019sec
    inet6 2401:4900:50a2:be84:a19f:f9af:e1a4:d2ef/64 scope global temporary dynamic 
       valid_lft 3483sec preferred_lft 3483sec
    inet6 2401:4900:50a2:be84:beed:9c45:6c2d:11ce/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3483sec preferred_lft 3483sec
    inet6 fe80::9fc:90cb:bcce:5587/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0:  mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:30:6a:47:21 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

We have loop back interface and then main interface connecting us to our home network. But what is docker0?

Docker0 is a new virtual bridge interface. It is the default interface and network for the default bridge (the default network and docker).
Let's run this command, this will list our current docker networks.

sudo docker network ls

The output is something like this

NETWORK ID     NAME      DRIVER    SCOPE
39e69177fb39   bridge    bridge    local
0b4b4acc5aa4   host      host      local
4717db76952b   none      null      local

We know "bridge" right here. But we don't know host and null. We will have a look at them soon. They are two of the seven networks in docker.
Notice the section called Driver. Driver basically means network type. So our bridge network is named bridge and its type is also bridge.

Now what if we deploy some containers in our default network?
We will use this command

sudo docker run -itd --rm --name container1 busybox

sudo docker run -itd --rm --name container2 busybox

sudo docker run -itd --rm --name container3 nginx

To make sure that the images are up we use this command

sudo docker ps

Output:-

CONTAINER ID   IMAGE     COMMAND                  CREATED              STATUS              PORTS     NAMES
9572a7554911   nginx     "/docker-entrypoint.…"   About a minute ago   Up About a minute   80/tcp    container3
edf3d34520ae   busybox   "sh"                     About a minute ago   Up About a minute             container2
0b090aed1589   busybox   "sh"                     2 minutes ago        Up 2 minutes                  container1

Now, when we deployed our containers, did we specified anything about networking at all? No. We let it do it by default. So it got thrown into our default bridge network. When we deployed those containers in the default network, docker automatically created three virtual ethernet interfaces and connected it to the docker0 bridge. It kind of act like a switch. There is a virtual ethernet interface for each container.
If we run ip address show again, it will show three new interfaces.

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlp2s0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether cc:6b:1e:5a:7b:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.52.189/24 brd 192.168.52.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 2232sec preferred_lft 2232sec
    inet6 2401:4900:50a2:be84:a19f:f9af:e1a4:d2ef/64 scope global temporary dynamic 
       valid_lft 3198sec preferred_lft 3198sec
    inet6 2401:4900:50a2:be84:beed:9c45:6c2d:11ce/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3198sec preferred_lft 3198sec
    inet6 fe80::9fc:90cb:bcce:5587/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0:  mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:30:6a:47:21 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:30ff:fe6a:4721/64 scope link 
       valid_lft forever preferred_lft forever
5: vethc88ce7d@if4:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 7e:15:d5:da:9e:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::7c15:d5ff:feda:9e7c/64 scope link 
       valid_lft forever preferred_lft forever
7: veth52546bb@if6:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 82:4b:fc:10:44:97 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::804b:fcff:fe10:4497/64 scope link 
       valid_lft forever preferred_lft forever
9: vethb942cb5@if8:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether e2:9f:02:22:df:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::e09f:2ff:fe22:dfc4/64 scope link 
       valid_lft forever preferred_lft forever

If we run this command, it will show there name and that they are connect to docker0

bridge link

Output:-

5: vethc88ce7d@if4:  mtu 1500 master docker0 state forwarding priority 32 cost 2 
7: veth52546bb@if6:  mtu 1500 master docker0 state forwarding priority 32 cost 2 
9: vethb942cb5@if8:  mtu 1500 master docker0 state forwarding priority 32 cost 2

Bridge not only created virtual ethernet interfaces, but also allocated IP addresses. Let's dive into the bridge network with this command

sudo docker inspect bridge

Output:-

[
    {
        "Name": "bridge",
        "Id": "39e69177fb39d5f46405f0b2bb2eb30f78397756fa88a623efb36845c78929ba",
        "Created": "2022-08-17T11:31:01.690732686+05:30",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "0b090aed15894615edc971e0d7fc717215507f8ef282710985e84425022ebc4f": {
                "Name": "container1",
                "EndpointID": "5c7e3f3751f513740f43982d263889042d08137fe0ea01fc86a71fc6e2545b40",
                "MacAddress": "02:42:ac:11:00:02",
                "IPv4Address": "172.17.0.2/16",
                "IPv6Address": ""
            },
            "9572a7554911d4ac23164d856484f48c39e6a81a3fb85cd5d9ef849303a7d832": {
                "Name": "container3",
                "EndpointID": "61c524a18b8803e077f0b3e765c23c8d5fc58524c352686802d17f2f80e39c5a",
                "MacAddress": "02:42:ac:11:00:04",
                "IPv4Address": "172.17.0.4/16",
                "IPv6Address": ""
            },
            "edf3d34520ae79ec97fcaa637708d1fd5f507954eeb33836cc35587ea01f70a1": {
                "Name": "container2",
                "EndpointID": "c1c66915c533d9dc95d2168c53deeb24e832a912e541539cbaadb9a55cb9544e",
                "MacAddress": "02:42:ac:11:00:03",
                "IPv4Address": "172.17.0.3/16",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

You can see that each container has its own ip address in that same docker0 network. It has DNS. And because the docker0 acts like a switch, the containers can talk to each other.

We installed nginx which by default is a website and it will use port 80. Now, can we reach that website? In other words, can my computer can access that web server?
The answer is no. This is the limitation of the bridge network. If you want to access any of the services that your docker containers offer (website in this case), it won't work by default. You have to manually expose those ports.
So let's expose port 80 to allow us to access and after that we have to redeploy container3. First remove the container3 with the help of this command

sudo docker stop container3

Then redeploy it with but this time by doing port mapping, telling docker to expose the container's port 80 to the host port 80.

sudo docker run -itd --rm -p 80:80 --name container3 nginx

Let's run this command to check what ports are being exposed

sudo docker ps

Output:-

CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS                               NAMES
252c2de84f26   nginx     "/docker-entrypoint.…"   50 seconds ago   Up 49 seconds   0.0.0.0:80->80/tcp, :::80->80/tcp   container3
edf3d34520ae   busybox   "sh"                     53 minutes ago   Up 53 minutes                                       container2
0b090aed1589   busybox   "sh"                     54 minutes ago   Up 54 minutes

Now we can see that now the website is running on our host machine.

It is not recommended to use default network but instead make you own network.

2) User-Defined Bridge

This network is pretty much exactly like the default bridge. The name itself suggests that you're making it. You have to create your own network.
Let's create our own network. Run this command

sudo docker network created tom

And that's it. Network created
If we check the list of docker networks by running command sudo docker network ls, it will show a network named tom

NETWORK ID     NAME      DRIVER    SCOPE
39e69177fb39   bridge    bridge    local
0b4b4acc5aa4   host      host      local
4717db76952b   none      null      local
4a17276f3a0f   tom       bridge    local

If we run ip address show, we will see a new virtual bridge is being created with a new network.

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlp2s0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether cc:6b:1e:5a:7b:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.52.189/24 brd 192.168.52.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 2303sec preferred_lft 2303sec
    inet6 2401:4900:50a2:be84:a19f:f9af:e1a4:d2ef/64 scope global temporary dynamic 
       valid_lft 3430sec preferred_lft 3430sec
    inet6 2401:4900:50a2:be84:beed:9c45:6c2d:11ce/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3430sec preferred_lft 3430sec
    inet6 fe80::9fc:90cb:bcce:5587/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0:  mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:30:6a:47:21 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:30ff:fe6a:4721/64 scope link 
       valid_lft forever preferred_lft forever
5: vethc88ce7d@if4:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 7e:15:d5:da:9e:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::7c15:d5ff:feda:9e7c/64 scope link 
       valid_lft forever preferred_lft forever
7: veth52546bb@if6:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 82:4b:fc:10:44:97 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::804b:fcff:fe10:4497/64 scope link 
       valid_lft forever preferred_lft forever
11: vethb9e82de@if10:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 96:29:18:b8:e1:21 brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::9429:18ff:feb8:e121/64 scope link 
       valid_lft forever preferred_lft forever
12: br-4a17276f3a0f:  mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:de:2b:a2:38 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-4a17276f3a0f
       valid_lft forever preferred_lft forever

Let's run a few containers inside this network

sudo docker run -itd --rm --network tom --name loki busybox

sudo docker run -itd --rm --network tom --name jerry busybox

Now if we do ip address show, we see we have virtual interfaces created.

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlp2s0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether cc:6b:1e:5a:7b:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.52.189/24 brd 192.168.52.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 1841sec preferred_lft 1841sec
    inet6 2401:4900:50a2:be84:a19f:f9af:e1a4:d2ef/64 scope global temporary dynamic 
       valid_lft 3565sec preferred_lft 3565sec
    inet6 2401:4900:50a2:be84:beed:9c45:6c2d:11ce/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3565sec preferred_lft 3565sec
    inet6 fe80::9fc:90cb:bcce:5587/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0:  mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:30:6a:47:21 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:30ff:fe6a:4721/64 scope link 
       valid_lft forever preferred_lft forever
5: vethc88ce7d@if4:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 7e:15:d5:da:9e:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::7c15:d5ff:feda:9e7c/64 scope link 
       valid_lft forever preferred_lft forever
7: veth52546bb@if6:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 82:4b:fc:10:44:97 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::804b:fcff:fe10:4497/64 scope link 
       valid_lft forever preferred_lft forever
11: vethb9e82de@if10:  mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 96:29:18:b8:e1:21 brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::9429:18ff:feb8:e121/64 scope link 
       valid_lft forever preferred_lft forever
12: br-4a17276f3a0f:  mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:de:2b:a2:38 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-4a17276f3a0f
       valid_lft forever preferred_lft forever
    inet6 fe80::42:deff:fe2b:a238/64 scope link 
       valid_lft forever preferred_lft forever
14: veth61ad963@if13:  mtu 1500 qdisc noqueue master br-4a17276f3a0f state UP group default 
    link/ether 62:ef:e3:9c:0f:6d brd ff:ff:ff:ff:ff:ff link-netnsid 3
    inet6 fe80::60ef:e3ff:fe9c:f6d/64 scope link 
       valid_lft forever preferred_lft forever
16: veth133cc0f@if15:  mtu 1500 qdisc noqueue master br-4a17276f3a0f state UP group default 
    link/ether 3e:be:1f:7a:b6:7c brd ff:ff:ff:ff:ff:ff link-netnsid 4
    inet6 fe80::3cbe:1fff:fe7a:b67c/64 scope link 
       valid_lft forever preferred_lft forever

And if we look at our bridge link, we see those new interfaces connected to that new virtual bridge.

5: vethc88ce7d@if4:  mtu 1500 master docker0 state forwarding priority 32 cost 2 
7: veth52546bb@if6:  mtu 1500 master docker0 state forwarding priority 32 cost 2 
11: vethb9e82de@if10:  mtu 1500 master docker0 state forwarding priority 32 cost 2 
14: veth61ad963@if13:  mtu 1500 master br-4a17276f3a0f state forwarding priority 32 cost 2 
16: veth133cc0f@if15:  mtu 1500 master br-4a17276f3a0f state forwarding priority 32 cost 2

We can inspect that network using sudo docker inspect tom we get:

[
    {
        "Name": "tom",
        "Id": "4a17276f3a0fbf55ca184c6eaf03016f88629c92a172647471429d3c3e9f51e3",
        "Created": "2022-08-17T20:31:12.910872226+05:30",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "1e2dfd713e0b43a500b8064464b8d506ff03627f6d3e59ab2ad369bd7395a7e1": {
                "Name": "loki",
                "EndpointID": "75751a3622caac5f8e1b2c85b860a4bfd01b9c6c1f4a6dc49105ae7bd9ad2df5",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            },
            "d3b48638734122961e50a72ee82dde566906d6aafae7b112f21ff45aa5bc2af1": {
                "Name": "jerry",
                "EndpointID": "dab02cd5bc96fb657faacb0dbcf144ffef920c6284a97ff09ad724c8107102f7",
                "MacAddress": "02:42:ac:12:00:03",
                "IPv4Address": "172.18.0.3/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

We prefer this network over default network. Reason? Because of the isolation. This tom network is isolated/protected from the default network. They can't talk to each other. Network isolation is very important. You have to isolate your workloads and this is why docker recommends a user-defined bridge.

3) Host network

We created container3 operating at port 80. Let's delete it and redeploy it on a host network.
Stop container by this command: sudo docker stop container3, then run this command

sudo docker run -itd --rm --network host --name container3 nginx

When you deploy a container to the host network, it doesn't even really have its own network. It bums off the host. It shares its IP address and ports. This means that you don't have to expose any ports. Now nginx can run as a regular application on the host without port mapping, even though it is inside a container. The downtime of this network is that there is no isolation. It's like right there next to the host.

4) Macvlan network

What if we could erase all of these stuff, all the complications, all the docker networks and the virtual ethernet interfaces, the separate networks and simply connect the docker containers directly to our physical network. That's what macvlan about. It would be like their ethernet interfaces are connected directly to the switch in my house. They even get thier own mac addresses. Also they will have their own IP addresses on my home network. They are acting like virtual machines.
Let's create our macvlan network.

sudo docker network create -d macvlan --subnet 192.168.239.0/24 --gateway 192.168.239.12 -o parent=wlp2s0 macvlan

What is parent=wlp2s0?
Here we have to tie our macvlan to our host network interface. The main interface of my ubuntu machine is wlp2s0. Check yours by doing ip address show. You must tie your macvlan network to a physical network.

Now lets do sudo docker network ls

NETWORK ID     NAME      DRIVER    SCOPE
ed28e5c4ce86   bridge    bridge    local
0b4b4acc5aa4   host      host      local
10d74c0e21fe   macvlan   macvlan   local
4717db76952b   none      null      local
4a17276f3a0f   tom       bridge    local

Here we go. We can see a new network named macvlan of type macvlan is created.
Now let's run some containers inside macvlan network.

sudo docker run -itd --rm --network macvlan --ip 192.168.239.23 --name jerry busybox

Now jerry is connected to macvlan network like a regular virtual machine.
In macvlan network, all the docker containers get their own mac addresses. Now where that becomes an issue is your network may not be able to have multiple mac addresses on one switch port. The cable connection is sharing a port with the host. They're all connected to the same port which will see multiple mac addresses and a lot of times port can't handle that. It might have port security, which says you can have one or maybe two mac addresses on one port and that breaks things. You will often see this called promiscuous mode. If you want to try this, you will need that enabled.

Let's first try with the host. We have to actually enable promiscuous mode on its network interface.
Run this command:

sudo ip link set wlp2s0 promisc on

Now lets see if it worked. Run this command and ping 192.168.239.23

sudo docker exec -it jerry sh

The macvlan has all the benefits of a bridge network, except its directly connected to your home network. If we deploy a web server in a macvlan, we don't have to expose any ports.

Macvlan has its downsides. That whole mac address thing happened to have promiscuous mode, which you may have no control over and the IP address thing, no dhcp. You would expect that if you connect your device directly to your home network, it will get an IP address from your router whatever that's offering dhcp. But no. If you don't specify an IP address when you deploy your container, docker will choose one for you. Docker will use its own dhcp and assign like 192.168.238.4 or 192.168.238.5 or anything which could create a litter conflict because you got two dhcp servers in your network. So in that scenario, just specify your IP address with every container.

Now let's talk about our next network.

5) Ipvlan network

This one solves the problem with macvlans, that promiscuous stuff. Ipvlan network has two different modes, L2 and L3. We will focus on L2 because it's pretty much same as macvlan with one difference and it solves our problem.
We have jerry and loki in a macvlan and they are assigned their own mac address. But it messes with our switches and the promiscuous stuff. Ipvlan allow the host to share its mac address with the containers. So the mac address of jerry and loki will match exactly to the host, but they will still have IP addresses on our network. This seems to resolve all issues.

Let's create ipvlan network

sudo docker network create -d ipvlan --subnet 192.168.239.0/24 --gateway 192.168.239.12 -o parent=wlp2s0 ipvlan

Now add new container:

sudo docker run -itd --rm --network ipvlan --ip 192.168.239.20 --name kashish busybox

Containerizing Nginx server

Kashish Lakhara — Sat, 06 Aug 2022 15:23:35 GMT

In this docker tutorial, I'll demonstrate how to containerize Nginx application. If you are a beginner, then you will find this tutorial very helpful. Even if you are not familiar with what Nginx is, it will be easy for you to follow this tutorial.

Prerequisites

1) To know the basics of docker you can check out my previous blog here
2) Docker Installation
For windows
For Mac
For ubuntu

To check if docker is installed successfully, run docker run -it hello-world. If it runs, then docker is successfully installed on your machine. You can check its version by docker -v.

Let's build container!

In this Docker tutorial, we will containerize Nginx application.
Create a dockerfile (dockerfiles are the files with no extension).

Some Dockerfile commands:-
Dockerfile is just a series of commands.

FROM It defines the base image to start the build process.

RUN If you want to run a particular image or a particular command, them you use this command.

CMD Similar to run, can be used for executing a specific command. It is not executed during build, but when a container is instantiated using the image being built.

ENTRYPOINT Suggests that when you've finished building your docker image, then the command which is specified with the entrypoint that will executed first when you run the docker container of that particular image.

ADD ADD or COPY command are the commands which can be interchangeably because the add command is used to copy whatever files are there in one directory to another directory.

ENV If an application needs a particular environment variable, then we can specify to our docker container that this application needs certain environment variables and this environment variable is present over here.

WORKDIR Used to set where the command defined with CMD is to be executed.

EXPOSE You can specify a port number where you want your application to run inside the container.

MAINTAINER If you want to tag your name along with the image which you are building, then you use this command.

USER If you want a particular user to execute or to run a container, then you can use this user command and specify that user.

VOLUME If you have multiple containers which are hosting the same application, then you might want them all to use the same path. So this is the path where it can be present.

Dockerfile:-

FROM ubuntu:22.04
MAINTAINER Kashish

RUN apt-get update && apt-get install -y nginx
ADD index.html /usr/share/nginx/html/index.html

CMD ["nginx", "-g", "daemon off;"]
EXPOSE 80

FROM ubuntu:22.04 means that we are using ubuntu as a base image

MAINTAINER Kashish just tags my name

apt-get update will update apt-get repositories
apt-get install -y nginx will install nginx service

Then we have to add index.html file to /usr/share/nginx/html/index.html, i.e, copy html file inside the container.

CMD ["nginx", "-g", "daemon off;"] will start nginx service. Nginx uses the daemon off directive to run in the foreground. If daemon on, then application would be running in background. With daemon off, we can see the UI.

EXPOSE 80 means our nginx service will be hosted on port 80.
Note: This runs within the container. If you want to access it on your host machine then you have to do port mapping while starting this container.

Create a simple html file.
HTML file:-

<html>
<head>
<title> Nginx Image </title>
</head>
<body>
<h1> Hello World!</h1>
</body>
</html>

Build an image and run the container

To build the docker image out of dockerfile, move to the directory where the dockerfile is saved and run sudo docker build -t nginximage . Wait for all the steps to be executed. This will be our custom ubuntu image.

We can now use this image to run the container.
To spin up a container out of this docker image run the following command.

sudo docker run -p 80:80 --name=container nginximage

Now your container would be active. Let's go to localhost and check if that's working.

To stop the container, run the command sudo docker ps. It will show your container id. Copy that and run sudo docker stop (container_id)

That's all for this tutorial. We you have any questions feel free to ask them in comment section. And as always your feedbacks are appreciated!

Introduction to Docker

Kashish Lakhara — Wed, 03 Aug 2022 03:50:08 GMT

Problems before Docker:-
A developer develops and runs an application. Now that application works in developer’s laptop but not in testing or production. This problem is arised due to difference in computing environment between developer, test and production team. In development environment there can be a software that is upgraded and in testing and production environment, the older version of software might be present.

What is Docker?
Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Docker containers are lightweight alternatives to Virtual Machines and it uses the host OS. You don't have to pre-allocate any RAM in containers.

Dockerfile builds a docker image and that image contains all the project's code. You can run that image to create as many docker containers as you want. Then this image can be uploaded on DockerHub, from DockerHub anyone can pull the image and build a container.

Docker images are huge in size and requires a lot of network bandwidth. So in order to save that network bandwidth, we use Jenkins server or any continuous integration server to build an environment that contains all the dependencies for a particular application or a microservice and that build environment is deployed onto various teams like testing, staging and production.

Create complex requirements for a microservice within an easy-to-write Dockerfile. Then push the code to the Git repository. CI server pull it down and build the exact environment that will be used in production to run the test suite without needing to configure the CI server at all. Deploy it out to a staging environment for testers. Roll exactly what you had in development, testing, and staging into production.

What is Docker registry?
Docker registry is a storage component for Docker images. We can store the images in either public or private repositories. DockerHub is famous public cloud repository.
Why use Docker Registries?
You can control where your images are being stored. Integrate image storage with your in-house development workflow.

What are Docker images?
Docker images are read only template that can be used to create containers out of it. These docker images contains all the dependencies for a particular application or any microservice. You can create your own image and upload that onto the DockerHub and at the same time you can also pull the images which are available in the public repositories.

What are Docker containers?
Docker containers are nothing but the runtime instances of docker images. It contains everthing that is required to run an application or any microservice. It is also possible that more than one image is required to create a container. Docker containers can be referred to as isolated application platform built from one or more docker images.

Docker compose
Suppose you have multiple applications on various containers and all those containers are linked together. So you don’t want to actually execute each of those containers one by one but want to run those containers at once with a single command. So here docker compose comes into picture.

Docker compose makes it easier to configure and run applications made up of multiple containers. For example, you define four containers in one YAML file and then run those four containers with a single command.

That's it for this one.
Thanks for reading!
Any feedbacks are welcome!!

What is a Protocol?

Kashish Lakhara — Tue, 26 Jul 2022 16:05:56 GMT

Human Analogy
It is probably easiest to understand the notion of a computer network protocol by first considering some human analogies.

Consider what you do when you want to ask someone for the time of day. Human protocol dictates that one first offer a greeting to initiate communication with someone else.. The typical response to a “Hi” is a returned “Hi” message. Implicitly, one then takes a cordial “Hi” response as an indication that one can proceed and ask for the time of day.

A different response to the initial “Hi” (such as “Don’t bother me”) might indicate an unwillingness or inability to communicate. In this case, the human protocol would be not to ask for the time of day. Sometimes one gets no response at all to a message, in such case one typically gives up asking that person for time.

Clearly, transmitted and received messages, and actions taken when these messages are sent or received or other events occur, play a central role in a human protocol. If people run different protocols (For example, if one person has manners but the other does not, or if one understands the concept of time and other does not) the protocols do not interoperate and no useful work can be accomplished. The same is true in networking – it takes two or more communicating entities running the same protocol in order to accomplish a task.

Network Protocols

A network protocol is similar to a human protocol, except that the entities exchanging messages and taking actions are hardware or software components of some device (like laptop, tablet, smartphone, router or other network-capable device). All the activity in the Internet that involves two or more communicating remote entities is governed by a protocol.

As an example of a computer network protocol with which you are probably familiar, consider what happens when you type the URL of a Web page into your Web browser. The scenario is illustrated in the right half of the figure.

First your machine will send a connection request message to the Web server and wait for a reply. The Web server will eventually receive your connection request message and return a connection reply message. Knowing that it is OK to request the Web document, your machine then sends the name of the Web page it wants to fetch from that Web server in a GET message. Finally, the web server returns the web page (file) to your machine.

Conclusion

Given the human and networking examples above, the exchange of messages and the actions taken when these messages are sent and received are the key defining elements of a protocol:
A protocol defines the format and the order of messages exchanged between two or more communicating entities, as well as the actions taken on the transmission and/or receipt of a message or other event.

The Internet, and computer networks in general, make extensive use of protocols. Different protocols are used to accomplish different tasks. Some protocols are simple and straightforward, while others are complex and intellectually deep.