Home:ALL Converter>Finding the layers and layer sizes for each Docker image

Finding the layers and layer sizes for each Docker image

Ask Time:2015-04-17T18:26:12         Author:user134589

Json Formatter

For research purposes I'm trying to crawl the public Docker registry ( https://registry.hub.docker.com/ ) and find out 1) how many layers an average image has and 2) the sizes of these layers to get an idea of the distribution.

However I studied the API and public libraries as well as the details on the github but I cant find any method to:

  • retrieve all the public repositories/images (even if those are thousands I still need a starting list to iterate through)
  • find all the layers of an image
  • find the size for a layer (so not an image but for the individual layer).

Can anyone help me find a way to retrieve this information?

EDIT: is anyone able to verify that searching for '*' in Docker registry is returning all the repositories and not just anything that mentions '*' anywhere? https://registry.hub.docker.com/search?q=\*

Author:user134589,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/29696656/finding-the-layers-and-layer-sizes-for-each-docker-image
Viswesn :

You can find the layers of the images in the folder /var/lib/docker/aufs/layers; provide if you configured for storage-driver as aufs (default option) \n\nExample:\n\n docker ps -a\n CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n 0ca502fa6aae ubuntu \"/bin/bash\" 44 minutes ago Exited (0) 44 seconds ago DockerTest\n\n\nNow to view the layers of the containers that were created with the image \"Ubuntu\"; go to /var/lib/docker/aufs/layers directory and cat the file starts with the container ID (here it is 0ca502fa6aae*)\n\n root@viswesn-vm2:/var/lib/docker/aufs/layers# cat 0ca502fa6aaefc89f690736609b54b2f0fdebfe8452902ca383020e3b0d266f9-init \n d2a0ecffe6fa4ef3de9646a75cc629bbd9da7eead7f767cb810f9808d6b3ecb6\n 29460ac934423a55802fcad24856827050697b4a9f33550bd93c82762fb6db8f\n b670fb0c7ecd3d2c401fbfd1fa4d7a872fbada0a4b8c2516d0be18911c6b25d6\n 83e4dde6b9cfddf46b75a07ec8d65ad87a748b98cf27de7d5b3298c1f3455ae4\n\n\nThis will show the result of same by running \n\nroot@viswesn-vm2:/var/lib/docker/aufs/layers# docker history ubuntu\nIMAGE CREATED CREATED BY SIZE COMMENT\nd2a0ecffe6fa 13 days ago /bin/sh -c #(nop) CMD [\"/bin/bash\"] 0 B \n29460ac93442 13 days ago /bin/sh -c sed -i 's/^#\\s*\\ (deb.*universe\\)$/ 1.895 kB \nb670fb0c7ecd 13 days ago /bin/sh -c echo '#!/bin/sh' > /usr/sbin/polic 194.5 kB \n83e4dde6b9cf 13 days ago /bin/sh -c #(nop) ADD file:c8f078961a543cdefa 188.2 MB \n\n\nTo view the full layer ID; run with --no-trunc option as part of history command.\n\ndocker history --no-trunc ubuntu\n",
2015-07-23T11:43:20
030 :

In my opinion, docker history <image> is sufficient. This returns the size of each layer:\n\n$ docker history jenkinsci-jnlp-slave:2019-1-9c\nIMAGE CREATED CREATED BY SIZE COMMENT\n93f48953d298 42 min ago /bin/sh -c #(nop) USER jenkins 0B\n6305b07d4650 42 min ago /bin/sh -c chown jenkins:jenkins -R /home/je… 1.45GB\n",
2019-01-09T10:53:36
lvthillo :

This will inspect the docker image and print the layers:\n\n$ docker image inspect nginx -f '{{.RootFS.Layers}}'\n[sha256:d626a8ad97a1f9c1f2c4db3814751ada64f60aed927764a3f994fcd88363b659 sha256:82b81d779f8352b20e52295afc6d0eab7e61c0ec7af96d85b8cda7800285d97d sha256:7ab428981537aa7d0c79bc1acbf208c71e57d9678f7deca4267cc03fba26b9c8]\n",
2018-05-15T19:04:57
sunnycmf :

They have a very good answer here:\nhttps://stackoverflow.com/a/32455275/165865\n\nJust run below images:\n\ndocker run --rm -v /var/run/docker.sock:/var/run/docker.sock nate/dockviz images -t\n",
2016-01-15T06:29:24
resultsway :

one more tool : https://github.com/CenturyLinkLabs/dockerfile-from-image\n\nGUI using ImageLayers.io ",
2015-07-29T02:00:22
Piet :

\nhttps://hub.docker.com/search?q=* shows all the images in the entire Docker hub, it's not possible to get this via the search command as it doesnt accept wildcards.\nAs of v1.10 you can find all the layers in an image by pulling it and using these commands:\n\ndocker pull ubuntu\nID=$(sudo docker inspect -f {{.Id}} ubuntu)\njq .rootfs.diff_ids /var/lib/docker/image/aufs/imagedb/content/$(echo $ID|tr ':' '/')\n\n\n\n3) The size can be found in /var/lib/docker/image/aufs/layerdb/sha256/{LAYERID}/size although LAYERID != the diff_ids found with the previous command. For this you need to look at /var/lib/docker/image/aufs/layerdb/sha256/{LAYERID}/diff and compare with the previous command output to properly match the correct diff_id and size.",
2016-03-14T10:53:43
Kane :

It's indeed doable to query the manifest or blob info from docker registry server without pulling the image to local disk.\n\nYou can refer to the Registry v2 API to fetch the manifest of image.\n\nGET /v2/<name>/manifests/<reference>\n\n\nNote, you have to handle different manifest version. For v2 you can directly get the size of layer and digest of blob. For v1 manifest, you can HEAD the blob download url to get the actual layer size.\n\nThere is a simple script for handling above cases that will be continuously maintained. ",
2020-04-29T03:59:25
Sandra Rossi :

To find all the layers of an image and to find the size for a layer, you can display the manifest from the docker hub registry via the "manifest" experimental feature:\ndocker manifest inspect ubuntu\n\nThe result is a JSON file (only the first lines are shown here):\n{\n "schemaVersion": 2,\n "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",\n "manifests": [\n {\n "mediaType": "application/vnd.docker.distribution.manifest.v2+json",\n "size": 529,\n "digest": "sha256:10cbddb6cf8568f56584ccb6c866203e68ab8e621bb87038e254f6f27f955bbe",\n "platform": {\n "architecture": "amd64",\n "os": "linux"\n }\n },\n {\n "mediaType": "application/vnd.docker.distribution.manifest.v2+json",\n "size": 529,\n "digest": "sha256:dd375524d7eda25a69f9f9790cd3e28855be7908e04162360dd462794035ebf7",\n "platform": {\n "architecture": "arm",\n "os": "linux",\n "variant": "v7"\n",
2021-09-10T20:25:56
Levon :

Check out dive written in golang. \n\n\n\nAwesome tool!",
2018-12-07T14:51:31
Oliver :

Not exactly the original question but to find the sum total of all the images without double-counting shared layers, the following is useful (ubuntu 18):\n\nsudo du -h -d1 /var/lib/docker/overlay2 | sort -h\n",
2020-02-07T23:45:22
Yuci :

You can first find the image ID using:\n$ docker images -a\n\nThen find the image's layers and their sizes:\n$ docker history --no-trunc <Image ID>\n\nNote: I'm using Docker version 1.13.1\n$ docker -v\nDocker version 1.13.1, build 092cba3\n",
2018-05-15T19:29:40
yy