Initial Stages

readme.md

@@ -0,0 +1,28 @@
+Under clusterwide-file-manifest we should have directory tree of the form
+
+dir - K-Y-Z-M 
+ K-Y-Z-M/clusterwide-manifest.nmd5
+
+These are the clusterwide manifests - i.e manifests that are produced by comparing the md5s of different nodes from the same cluster,
+and producing 2 lists for each of those clusters:
+
+List 1 (matches):
+md5sum:/file/path
+
+List 2 (diffs):
+md5sum:/file/path
+
+Where List 1 is a list of all files that have the same path, and have the same path and md5sum across the cluster
+While list 2 is a list of the inverse of the above.
+
+# Stages
+
+## Stage 1 - Md5:filepath
+ - Param 1 - base path (i.e `stage1.sh /` - would produce md5sum for everything under root)
+ - Produces a file containing the md5:file on a given system
+ - Excludes directories known to contain node-specific directories
+
+## Stage 2 - Base manifest producer -
+ Takes in files produced from Stage 1 across a cluster and produces an output base-manifest
+
+## Stage 3 - Compares a stage 1 output against a known stage 2 manifest