Update flag to include-latest-collection-only

Per some discussions with Steve and Co, by default, our software should include all the products and let NSSDCA figure out which ones they are already have. We should still provide the flag, but we just needed to change the default functionality.

refs #24

docs/source/usage/index.rst

@@ -30,8 +30,8 @@ Running ``pds-deep-archive --help`` will give a summary of the command-line
 invocation, its required arguments, and any options that refine the behavior.
 
 
-Example
--------
+Example 1: Basic Usage
+----------------------
 
 For example, to create a SIP and AIP from the LADEE 1101 Bundle located at
 ``test/data/ladee_test/mission_bundle/LADEE_Bundle_1101.xml`` run the
@@ -98,21 +98,38 @@ package (e.g. ``ladee_mission_bundle_v1.0_pds_deep_archive.zip``).
 Email the data package to the `PDS Operator <mailto:pds-operator@jpl.nasa.gov>`_.
 
 
+Example 2: Bundle Referencing Collections by LID
+------------------------------------------------
+
+For running the software on an accumulating bundle, you have two options:
+
+**Option 1: (preferred) Run ``pds-deep-archive`` against the entire bundle and all versions of all collections present beneath the bundle root directory.**
+
+This is the default functionality of ``pds-deep-archive``. See `Example 1: Basic Usage`_ to execute using this option.
+
+**Option 2: Run ``pds-deep-archive`` against the bundle with only the current versions of the collections present beneath the bundle root directory.**
+
+Execute the software with the ``--include-latest-collections-only`` flag enabled, for example::
+
+    (pds-deep-archive) $ pds-deep-archive -s PDS_ATM  \ 
+                            -b https://atmos.nmsu.edu/PDS/data/PDS4/LADEE/  \
+                            test/data/ladee_test/mission_bundle/LADEE_Bundle_1101.xml
+                            --include-latest-collection-only
+
+
 PDS Delivery Checklist
 ----------------------
 
 The following is a checklist and procedure Discipline Node personnel should
 follow when delivering a PDS Deep Archive data package to the PDS Engineering
 Node upon a new release of data:
 
-•  ▶ START: New Bundle is ready for Delivery.
-•  Bundle has completed successful validation with PDS4 Validate Tool.
-•  Execute PDS Deep Archive software per usage instructions and example above.
-•  Check the SIP Manifest (``*._sip_v1.0.tab``) file to verify URLs indicated
-   are valid.
-•  Package up the five ``*.tab` and ``*.xml`` files into a ``.ZIP`` or
-   ``.TAR.GZ`` PDS Deep Archive Delivery package.
-•  Email PDS Deep Archive Delivery package to the
-   `PDS Operator <mailto:pds-operator@jpl.nasa.gov>`_ for delivery to NSSDCA.
-•  Receive confirmation from PDS Operator once delivery has completed.
+•  ▶  START: New Bundle is ready for Delivery.
+•     Bundle has completed successful validation with PDS4 Validate Tool.
+•     Execute PDS Deep Archive software per usage instructions and example above.
+•     Check the SIP Manifest (``*._sip_v1.0.tab``) file to verify URLs indicated are valid.
+•     Package up the five ``*.tab` and ``*.xml`` files into a ``.ZIP`` or
+      ``.TAR.GZ`` PDS Deep Archive Delivery package.
+•     Email PDS Deep Archive Delivery package to the `PDS Operator <mailto:pds-operator@jpl.nasa.gov>`_ for delivery to NSSDCA.
+•     Receive confirmation from PDS Operator once delivery has completed.
 •  🎉 DONE

src/pds/aipgen/aip.py

@@ -388,7 +388,7 @@ def main():
         ts = datetime(ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second, microsecond=0, tzinfo=None)
 
         # Here we go, daddy
-        process(args.bundle, args.include_all_collections, con, ts)
+        process(args.bundle, args.args.include_latest_collection_only, con, ts)
     _logger.info('👋 Thanks for using this program! Bye!')
     sys.exit(0)
 

src/pds/aipgen/main.py

@@ -103,7 +103,7 @@ def main():
         ts = datetime.utcnow()
         ts = datetime(ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second, microsecond=0, tzinfo=None)
 
-        chksumFN, dummy, dummy = aipProcess(args.bundle, args.include_all_collections, con, ts)
+        chksumFN, dummy, dummy = aipProcess(args.bundle, args.include_latest_collection_only, con, ts)
         with open(chksumFN, 'rb') as chksumStream:
             sipProcess(
                 args.bundle,
@@ -118,7 +118,7 @@ def main():
                 args.offline,
                 args.bundle_base_url,
                 chksumStream,
-                args.include_all_collections,
+                args.include_latest_collection_only,
                 con,
                 ts
             )

src/pds/aipgen/sip.py

@@ -481,7 +481,7 @@ def main():
             offline=args.offline,
             baseURL=args.bundle_base_url,
             aipFile=args.aip,
-            allCollections=args.include_all_collections,
+            allCollections=args.include_latest_collection_only,
             con=con,
             timestamp=ts
         )

src/pds/aipgen/utils.py

@@ -48,9 +48,10 @@
 _digestCacheSize = 2**16                                 # Message digests to cache in memory
 _pLineMatcher = re.compile(r'^P,\s*([^\s]+)::([^\s]+)')  # Match separate lids and vids in "P" lines in ``.tab`` files
 
-# Help message for ``--include-all-collections``:
-_allCollectionsHelp = '''For bundles that reference collections by LID, this flag will include ALL versions of
-collections in the bundle. By default, the software only includes the latest version of the collection.
+# Help message for ``--include-latest-collection-only``:
+_allCollectionsHelp = '''For bundles that reference collections by LID, this flag will only include the latest version of
+collections in the bundle. By default, the software includes all versions of all collections located within the bundle 
+root directory.
 '''
 
 
@@ -218,4 +219,4 @@ def addBundleArguments(parser):
     '''Add command-line parsing to the given argument ``parser`` to support handling of bundles
     with ambiguous ``lid_reference`` without specific versions (#24)
     '''
-    parser.add_argument('--include-all-collections', action='store_true', default=False, help=_allCollectionsHelp)
+    parser.add_argument('--include-latest-collection-only', action='store_false', default=True, help=_allCollectionsHelp)