Skip to content

Commit

Permalink
Add docker2sqlelf script
Browse files Browse the repository at this point in the history
Introduce a script to create sqlelf (sqlite) files for the distribution
easily based on a Docker image.
  • Loading branch information
fzakaria committed Mar 10, 2024
1 parent 85efdff commit ebaf7cf
Show file tree
Hide file tree
Showing 2 changed files with 123 additions and 0 deletions.
45 changes: 45 additions & 0 deletions tools/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Tools

This folder contains various tools that can be used in conjunction with sqlelf.

## [docker2sqlelf](./docker2sqlelf)

This script can be given a Docker image name and produce a single sqlite file, using the sqlelf schema, containing
all ELF libraries and binaries found within the image.

```console
$ ./tools/docker2sqlelf debian:stable-20240211
```

A lot of interesting analysis becomes possible when you have a single sqlite file to work with.
You can even attach multiple distributions together and query across them.

```console
$ sqlite3 debian.sqlite
> ATTACH './debian-stable-20240211.sqlite' AS 'debian-stable-20240211';
> ATTACH './debian-stable-20230612.sqlite' AS 'debian-stable-20230612';
> SELECT * FROM pragma_database_list;
seq name file
--- ---------------------- ------------------------------------------------------------
0 main /home/fzakaria/sqlelf/debian.sqlite

2 debian-stable-20240211 /home/sqlelf/debian-stable-20240211.sqlite

3 debian-stable-20230612 /home/fzakaria/sqlelf/debian-stable-20230612.sqlite
```

You can run interesting queries across multiple distributions.

For instance, you can find whether any supported GLIBC version has been added or removed across the two distributions.

```sql
SELECT version
FROM (
SELECT version, 20230612 AS debian_version FROM `debian-stable-20230612`.elf_symbols
UNION ALL
SELECT version, 20240211 AS debian_version FROM `debian-stable-20240211`.elf_symbols
)
WHERE version LIKE 'GLIBC\_%' ESCAPE '\'
GROUP BY version
HAVING COUNT(DISTINCT debian_version) = 1;
```
78 changes: 78 additions & 0 deletions tools/docker2sqlelf
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/usr/bin/env bash

set -e -o pipefail

# Default behavior is to clean up the temporary directory
KEEP_TEMP_DIR=false

# Parse command-line arguments
while [[ "$#" -gt 0 ]]; do
case $1 in
-k|--keep) KEEP_TEMP_DIR=true ;;
*) IMAGE_NAME="$1" ;;
esac
shift
done

# Ensure an image name was provided
if [ -z "$IMAGE_NAME" ]; then
echo "Usage: $0 [-k|--keep] <docker-image-name>"
exit 1
fi

# Create a temporary directory for the container's filesystem
TEMP_DIR=$(mktemp -d)
echo "Created temporary directory at $TEMP_DIR"

# Function to cleanup on exit, if not keeping the temporary directory
cleanup() {
if [ "$KEEP_TEMP_DIR" = false ]; then
echo "Cleaning up..."
# Remove temporary directory
rm -rf "$TEMP_DIR"
echo "Removed temporary directory $TEMP_DIR"
else
echo "Keeping temporary directory $TEMP_DIR"
fi
}

# Create a temporary directory for the container's filesystem
TEMP_DIR=$(mktemp -d)
echo "Created temporary directory at $TEMP_DIR"s

# Trap EXIT to cleanup
trap cleanup EXIT

# Create docker container from the image
CONTAINER_ID=$(docker create "$IMAGE_NAME")
echo "Created container with ID $CONTAINER_ID"

# Export the container's filesystem and untar it to the temporary directory
docker export "$CONTAINER_ID" | tar -C "$TEMP_DIR" -xf -
echo "Exported and extracted container's filesystem to $TEMP_DIR"

# Remove the docker container
docker rm "$CONTAINER_ID"
echo "Removed container $CONTAINER_ID"

# Replace colons with hyphens
MODIFIED_IMAGE_NAME="${IMAGE_NAME//:/-}"

# Uncomment the below lines if this script is failing on a distribution
# it will run sqlelf on each file one at a time, and it will tell you
# which file it is failing on
# echo "Running sqlelf on the contents of $TEMP_DIR one file at a time."
# KEEP_TEMP_DIR=true
# find "$TEMP_DIR" -type f -print0 | while IFS= read -r -d $'\0' file; do
# if ! file "$file" | grep -q 'ELF'; then
# continue
# fi
# echo "Processing $file with sqlelf"
# sqlelf "$file" --sql "SELECT 1;"
# done

# Run sqlelf tool on the contents of the temporary directory
echo "Running sqlelf on the contents of $TEMP_DIR"
sqlelf "$TEMP_DIR" --sql ".backup ${MODIFIED_IMAGE_NAME}.sqlite"

echo "Script completed successfully."

0 comments on commit ebaf7cf

Please sign in to comment.