added build test (#5)

Author: ayush-srivastava-03

CHANGELOG.md

@@ -1,8 +1,8 @@
-# Connector-Zenko Changelog
+# Storj-Zenko Changelog
 
-## [1.0.5] - 17-08-2020
+## [1.0.5] - 01-09-2020
 ### Changelog:
-* Resolved upload path issue.
+* Added test file for CI-CD pipeline.
 
 ## [1.0.5] - 22-05-2020
 ### Changelog:

Dockerfile

@@ -0,0 +1,5 @@
+FROM golang:1.15
+RUN mkdir /app
+COPY . /app/
+WORKDIR /app/cmd
+RUN go test -v main_test.go

README.assets/Home.md

@@ -2,25 +2,26 @@
 
 ![](https://github.com/storj-thirdparty/connector-zenko/blob/master/README.assets/arch.drawio.png)
 
-
 ## Config Files
+
 There are two config files that contain Storj network and Zenko connection information. The tool is designed so you can specify a config file as part of your tooling/workflow.
 
-##### ```zenko_property.json```
-Inside the ```./config``` directory there is a ```zenko_property.json``` file, with following information about your Zenko instance:
+### `zenko_property.json`
+
+Inside the `/config` directory there is a `zenko_property.json` file, with following information about your Zenko instance:
 
 * zenkoEndpoint - S3 End point of Zenko Instance
 * accessKeyID - S3 Access Key ID created in Zenko Instance
 * secretAccessKey - S3 Secret Access Key created in Zenko Instance
 
-##### ```storj_config.json```
-Inside the ```./config``` directory a ```storj_config.json``` file, with Storj network configuration information in JSON format:
+### `storj_config.json`
+Inside the `./config` directory a `storj_config.json` file, with Storj network configuration information in JSON format:
 
 * apiKey - API Key created in Storj Satellite GUI
 * satelliteURL - Storj Satellite URL
 * encryptionPassphrase - Storj Encryption Passphrase.
 * bucket - Name of the bucket to upload data into.
-* uploadPath - Path on Storj Bucket to store data (optional) or "" or "/" (mandatory)
+* uploadPath - Path on Storj Bucket to store data (optional) or "/"
 * serializedAccess - Serialized access shared while uploading data used to access bucket without API Key
 * allowDownload - Set true to create serialized access with restricted download
 * allowUpload - Set true to create serialized access with restricted upload
@@ -30,48 +31,63 @@ Inside the ```./config``` directory a ```storj_config.json``` file, with Storj n
 * notAfter - Set time that is always after notBefore
 
 ## Run
+
 Zenko files are iterated through and streamed in 32KB chunks to the Storj network.
 
-The following flags can be used with the ```store``` command:
+The following flags can be used with the `store` command:
 
-* ```accesskey``` - Connects to the Storj network using a serialized access key instead of an API key, satellite url and encryption passphrase .
-* ```share``` - Generates a restricted shareable serialized access with the restrictions specified in the Storj configuration file.
+* `accesskey` - Connects to the Storj network using a serialized access key instead of an API key, satellite url and encryption passphrase.
+* `share` - Generates a restricted shareable serialized access with the restrictions specified in the Storj configuration file.
 
 Once you have built the project you can run the following:
 
-##### Get help
+### Get help
+
 ```
 $ ./connector-zenko --help
 ```
-##### Check version
+
+### Check version
+
 ```
 $ ./connector-zenko version
 ```
-##### Create backup from Zenko and upload them to Storj
+
+### Create backup from Zenko and upload them to Storj
+
 ```
 $ ./connector-zenko store --zenko <path_to_zenko_config_file> --storj <path_to_storj_config_file>
 ```
-##### Create backup files from Zenko and upload them to Storj bucket using Access Key
+
+### Create backup files from Zenko and upload them to Storj bucket using Access Key
+
 ```
 $ ./connector-zenko store --accesskey
 ```
-##### Create backup files from Zenko and upload them to Storj and generate a Shareable Access Key based on restrictions in ```storj_config.json```.
+
+### Create backup files from Zenko and upload them to Storj and generate a Shareable Access Key based on restrictions in `storj_config.json`.
+
 ```
 $ ./connector-zenko store --share
 ```
 		
 ## Testing
+
 * The project has been tested on the following operating systems:
+
 ```
 	* Windows
 		* Version: 10 Pro
 		* Processor: Intel(R) Core(TM) i3-5005U CPU @ 2.00GHz 2.00GHz
+		* Zenko version: v1.2.0
 
 	* macOS Catalina
 		* Version: 10.15.4
 		* Processor: 2.5 GHz Dual-Core Intel Core i5
+		* Zenko version: v1.2.0
 
 	* ubuntu
 		* Version: 16.04 LTS
 		* Processor: AMD A6-7310 APU with AMD Radeon R4 Graphics × 4
+		* Zenko version: v1.2.0
 ```		

README.md

@@ -4,14 +4,14 @@
 [![Go Report Card](https://goreportcard.com/badge/github.com/storj-thirdparty/connector-zenko)](https://goreportcard.com/report/github.com/storj-thirdparty/connector-zenko)
 ![Cloud Build](https://storage.googleapis.com/storj-utropic-services-badges/builds/connector-zenko/branches/master.svg)
 
-
 ## Overview
 
 Command line application (on Windows/Linux/Mac) for taking data backup from Zenko to Storj. Application connects to Zenko server and the souce code for interaction to Storj for cloud storage which is written in Golang.
 
 Zenko is infrastructure software to control Data in Multi-Cloud IT Environments without cloud lock-in and has features such as enabling unified data management from anywhere through a secure cloud portal, providing a single S3 endpoint through which data can be stored, retrieved and searched across any location.
 
 ### Features of connector-zenko:
+
 * Connects S3 compatible cloud storages (e.g. Amazon AWS, Azure Blob, Google Cloud Storage, Wasabi) to the Zenko instance for backing up their data to StorJ V3 network.
 * Upload any type of data from Zenko to Storj (single or multiple at once) whether it is a folder, document, data file, image, video, etc.
 ```
@@ -26,49 +26,63 @@ Available Commands:
 
 ```store``` - Connect to the specified Zenko instance (default: ```zenko_property.json```). Backups of the Zenko storage are generated using tooling provided by Zenko then uploaded to the Storj network. Connect to a Storj v3 network using the access specified in the Storj configuration file (default: ```storj_config.json```).
 
-
 Sample configuration files are provided in the ```./config``` folder.
 
 ## Requirements and Install
+
 To build from scratch, [install the latest Go](https://golang.org/doc/install#install).
 
 > Note: Ensure go modules are enabled (GO111MODULE=on)
 
 #### Option #1: clone this repo (most common)
+
 To clone the repo
+
 ```
 git clone https://github.com/storj-thirdparty/connector-zenko.git
 ```
+
 Then, build the project using the following:
+
 ```
 cd connector-zenko
 go build
 ```
+
 #### Option #2: go get into your gopath
+
 To download the project inside your GOPATH use the following command:
+
 ```
 go get github.com/storj-thirdparty/connector-zenko
 ```
+
 ## Run (short version)
+
 Once you have built the project run the following commands as per your requirement:
 
 ##### Get help
+
 ```
 $ ./connector-zenko --help
 ```
+
 ##### Check version
+
 ```
 $ ./connector-zenko --version
 ```
+
 ##### Create backup from Zenko and upload to Storj
+
 ```
 $ ./connector-zenko store
 ```
+
 ## Documentation
 
 For more information on runtime flags, configuration, testing, and diagrams, check out the [Detail](//github.com/storj-thirdparty/connector-zenko/wiki) or jump to:
 
-
 * [Config Files](//github.com/storj-thirdparty/connector-zenko/wiki/#config-files)
 * [Run (long version)](//github.com/storj-thirdparty/connector-zenko/wiki/#run)
 * [Testing](//github.com/storj-thirdparty/connector-zenko/wiki/#testing)

cloudbuild.yaml

@@ -1,29 +1,33 @@
 steps:
-  # see https://www.npmjs.com/package/editorconfig-checker
+  - name: gcr.io/cloud-builders/gcloud
+    entrypoint: 'bash'
+    args: ["-c","gcloud secrets versions access latest --secret=testFile >>testFile.txt" ]
+  - name: gcr.io/cloud-builders/gcloud
+    entrypoint: 'bash'
+    args: ["-c","gcloud secrets versions access latest --secret=connector_secret >>./config/storj_config_test.json" ]
+  - name: gcr.io/cloud-builders/docker
+    args: ['build', '-t', 'gcr.io/$PROJECT_ID/connector-zenko', '.']
   - id: eclint
     name: "e53e225/editorconfig-checker"  
   - id: go_version
     name: "gcr.io/cloud-builders/go"
     args: ["version"]
-    env: ["GOPATH=."]  
+    env: ["GOPATH=."]
   - id: go_linter
     name: "golangci/golangci-lint"
     args: ["golangci-lint","run"]  
-    env: ["GO111MODULE=on", "CGO_ENABLED=0"]
-  - id: go_security
-    name: "securego/gosec"
-    args: [ "./..." ] 
+    env: ["GO111MODULE=on", "CGO_ENABLED=0"] 
   - id: go_build
     name: "mirror.gcr.io/library/golang"
     env: ['GO111MODULE=on']
     args: ['go', 'build', './...']
   - id: doc_check
     name: "ubuntu"
     args: ['bash', './test/doc-check.sh']
-#   name: "gcr.io/cloud-builders/go"
-#   args: ["build"]
-#   env: ["GOPATH=.","GO111MODULE=on", "CGO_ENABLED=0"]
-# - id: go_test
-#   name: "gcr.io/cloud-builders/go"
-#   args: ["test","helloworld"]
-#   env: ["GOPATH=."]  
+  - id: go_security
+    name: "securego/gosec"
+    args: ["./..." ]
+
+tags: ['cloud-builders-community']
+images: ['gcr.io/$PROJECT_ID/connector-zenko']
+tags: ['cloud-builders-community']

cmd/main_test.go

@@ -0,0 +1,42 @@
+package cmd_test
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"os"
+
+	"testing"
+
+	"github.com/storj-thirdparty/connector-zenko/cmd"
+)
+
+func TestZenkoStore(t *testing.T) {
+
+	testFlag := true
+
+	storjConfig := cmd.LoadStorjConfiguration("../config/storj_config_test.json")
+	_, project := cmd.ConnectToStorj(storjConfig, false)
+
+	fileReader, err := os.Open("../testFile.txt")
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	fmt.Printf("Initiating back-up.\n")
+	cmd.UploadData(project, storjConfig, "testFile.txt", nil, fileReader, testFlag)
+	fmt.Printf("Back-up complete.\n\n")
+
+	fmt.Printf("\nDeleting the test back-up.\n")
+	ctx := context.Background()
+	backups := project.ListObjects(ctx, storjConfig.Bucket, nil)
+	// Loop to find the latest back-up of all the back-ups.
+	for backups.Next() {
+		item := backups.Item()
+		_, err := project.DeleteObject(ctx, storjConfig.Bucket, item.Key)
+		if err != nil {
+			log.Fatal(err)
+		}
+	}
+	fmt.Printf("Deleted the test back-up.\n")
+}

cmd/store.go

@@ -28,6 +28,8 @@ func init() {
 
 func zenkoStore(cmd *cobra.Command, args []string) {
 
+	testFlag := false
+
 	// Process arguments from the CLI.
 	zenkoConfigfilePath, _ := cmd.Flags().GetString("zenko")
 	fullFileNameStorj, _ := cmd.Flags().GetString("storj")
@@ -41,7 +43,7 @@ func zenkoStore(cmd *cobra.Command, args []string) {
 	storjConfig := LoadStorjConfiguration(fullFileNameStorj)
 
 	// Connect to storj network using the specified credentials.
-	access, project := ConnectToStorj(fullFileNameStorj, storjConfig, useAccessKey)
+	access, project := ConnectToStorj(storjConfig, useAccessKey)
 
 	// Establish connection with Zenko and get io.Reader implementor.
 	zenkoReader := ConnectToZenko(configZenko)
@@ -52,7 +54,7 @@ func zenkoStore(cmd *cobra.Command, args []string) {
 	fmt.Printf("\nInitiating back-up.\n")
 	// Fetch all backup files from Zenko instance and simultaneously store them into desired Storj bucket.
 	for i := 0; i < len(uploadPathNames); i++ {
-		UploadData(project, storjConfig, uploadPathNames[i], zenkoObjectReaders[i])
+		UploadData(project, storjConfig, uploadPathNames[i], zenkoObjectReaders[i], nil, testFlag)
 	}
 	fmt.Printf("\nBack-up complete.\n\n")