Update readme.md

readme.md

@@ -1,22 +1,32 @@
 Under clusterwide-file-manifest we should have directory tree of the form
 
-dir - K-Y-Z-M 
+`dir` - `K-Y-Z-M`:
- K-Y-Z-M/clusterwide-manifest.nmd5
+- `K-Y-Z-M`/`clusterwide-manifest.nmd5`
 
 These are the clusterwide manifests - i.e manifests that are produced by comparing the md5s of different nodes from the same cluster,
 and producing 2 lists for each of those clusters:
 
-List 1 (matches):
+- List 1 (matches):
 md5sum:/file/path
 
-List 2 (diffs):
+- List 2 (diffs):
 md5sum:/file/path
-
+> Where List 1 is a list of all files that have the same path, and have the same path and md5sum across the cluster
-Where List 1 is a list of all files that have the same path, and have the same path and md5sum across the cluster
+> While list 2 is a list of the inverse of the above.
-While list 2 is a list of the inverse of the above.
 
 # Stages
 
+## Stage 0 - ssh backbone for propogating stage1 across the cluster
+```bash
+yum install git -y
+git clone https://git.adspem.com/opthq/nmd-md5sum-manifest.git
+cd nmd-md5sum-manifest/stages
+chmod +x *
+./stage0-remote-run.sh
+hsctl cmd run 'ls -la /root/manifest*'
+./stage1-1-get-remote-files.sh
+```
+
 ## Stage 1 - Md5:filepath
  - Param 1 - base path (i.e `stage1.sh /` - would produce md5sum for everything under root)
  - Produces a file containing the md5:file on a given system

stages/stage0-remote-run.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 ssh_keyfile='/export/home/cloudian/cloudian-installation-key'
-script='/tmp/nmd/nmd-md5sum-manifest/stages/stage1-gather-node-manifest.sh'
+script='stage1-gather-node-manifest.sh'
 
 # Ensure environment variables are set
 if [[ -z "$ssh_keyfile" || -z "$script" ]]; then

stages/stage1-1-get-remote-files.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+ssh_keyfile='/export/home/cloudian/cloudian-installation-key'
+# Set the path to the remote file you want to download
+remote_file='/root/manifest-*txt*'
+
+# Ensure variables are set
+if [[ -z "$ssh_keyfile" || -z "$remote_file" ]]; then
+    echo "Error: Please set 'ssh_keyfile' and 'remote_file' variables."
+    exit 1
+fi
+
+# Extract hostnames/IPs from /etc/hosts, skipping comments and localhost
+HOSTS=$(grep -v '^#' /etc/hosts | grep -v 'localhost' | awk '{print $2}')
+
+for HOST in $HOSTS; do
+    echo "--- Fetching from: $HOST ---"
+
+    # Create a unique local directory for this host's file
+    #target_dir="remotes/$HOST"
+    target_dir="remotes/"
+    mkdir -p "$target_dir"
+
+    # Use SCP to pull the file from the remote host to the local directory
+    # -i: Specifies the identity (private key) file
+    # -o StrictHostKeyChecking=no: Skips manual fingerprint verification prompts
+    scp -i "$ssh_keyfile" -o StrictHostKeyChecking=no "$HOST":"$remote_file" "$target_dir/"
+
+    if [ $? -eq 0 ]; then
+        echo "--- Success: File saved to $target_dir/ ---"
+    else
+        echo "--- Failed: Could not fetch from $HOST ---"
+    fi
+done
+