Skip to content

Commit f6fe5d0

Browse files
ununhexiumununhexium
authored
Integrate the broker's extensions (#921)
* Add accidentally deleted SUMMARY * chore: Apply new config style * Code review * Merge docs * Merge trivial templates changes * Merge more conflicting files * Merge more conflicting files * Merge more conflicting files and cleanup old versions * Add connector submodule * Merge more conflicting files * Move broker CHANGELOG * Move the broker's README * More cleaning * Fix links * Prepare docker build to merge the broker * Merge the docker build * Fix links and format * Add connector type info * self review * Convert MD table to html * Only use test containers for the Broker * Add broker NPM delivery * code review * Add broker local dev doc * Merge deployment READMEs and change healthcheck header * Fix links * Cleanup * Fix javadocs and add -sources and -javadocs artefacts * Rename .env.extensions to .env.connector * Rename IDS broker -> broker * Remove redundant .editorconfig setting * Change connector-type -> deployment-type in workflow config * NPM clean install * Removed mention of the broker in the main readme * Dry postgres image name * Remove test DB deprecation * Move broker group to toml * Fix missing broker's groups * More versions cleanup * Change MD for HTML * Merge the broker compose file * #921 (comment) * Merge local quick start for broker * Align postgres versions * Stop annoying me with DB not ready messages on new startup * Wait for the DBs * Code review * remove docs task. The repo is now public * Avoid star imports * Remove reference to the merged doc
1 parent bc0e536 commit f6fe5d0

File tree

118 files changed

+1002
-3570
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

118 files changed

+1002
-3570
lines changed

‎.dockerignore

+5-1
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,8 @@ build
3838
*.hprof
3939

4040
.env
41-
!launchers/.env
41+
!launchers/.env.connector
42+
!launchers/.env.broker
4243

4344
# Log files
4445
*.log
@@ -48,3 +49,6 @@ build
4849
**/*.jks
4950

5051
docs/secrets
52+
53+
node_modules
54+
npm-debug.log

‎.editorconfig

+20
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,23 @@ end_of_line = lf
88
charset = utf-8
99
trim_trailing_whitespace = true
1010
insert_final_newline = true
11+
indent_size = 4
12+
13+
[*.ts]
14+
quote_type = single
15+
16+
[*.md]
17+
max_line_length = off
18+
trim_trailing_whitespace = false
19+
indent_size = 2
20+
21+
[*.{yml,yaml}]
22+
indent_size = 2
23+
24+
[{*.kt,*.kts}]
25+
ij_kotlin_imports_layout = *, java.**, javax.**, kotlin.**, ^
26+
ij_kotlin_packages_to_use_import_on_demand = explicitly-none
27+
ij_kotlin_name_count_to_use_star_import = 2147483647
28+
ij_kotlin_name_count_to_use_star_import_for_members = 2147483647
29+
ij_continuation_indent_size = 4
30+

‎.env

+1
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ EDC_IMAGE=ghcr.io/sovity/edc-dev:7.4.2
33
TEST_BACKEND_IMAGE=ghcr.io/sovity/test-backend:7.4.2
44
EDC_UI_IMAGE=ghcr.io/sovity/edc-ui:3.2.2
55
EDC_UI_ACTIVE_PROFILE=sovity-open-source
6+
BROKER_IMAGE=ghcr.io/sovity/broker-server-dev:4.2.0

‎.env.dev

+1-1
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ EDC_IMAGE=ghcr.io/sovity/edc-dev:latest
33
TEST_BACKEND_IMAGE=ghcr.io/sovity/test-backend:latest
44
EDC_UI_IMAGE=ghcr.io/sovity/edc-ui:latest
55
EDC_UI_ACTIVE_PROFILE=sovity-open-source
6-
6+
BROKER_IMAGE=ghcr.io/sovity/broker-server-dev:latest

‎.github/ISSUE_TEMPLATE/feature_request_mds.md

+8
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,14 @@ _What problems does that user face that existing functionalities do solve?_
2626

2727
_Describe whether this request is related to an existing workflow, feature, or otherwise something in the product today. Or, does this open us up to new innovative ideas?_
2828

29+
## MDS Scope check
30+
31+
_Is this feature part of the contracted scope?_
32+
- [ ] It is part of the contracted scope
33+
- [ ] It is not part of the contracted scope
34+
35+
If not, please add the label "mds/future-scope"
36+
2937
## (For sovity Team to complete) Stakeholders
3038

3139
_Add more on who asked for this, i.e. company, person, how much they pay us, what their tier is, are they a strategic account, etc. Who needs to be kept up-to-date about this feature?_

‎conflicts/.github/ISSUE_TEMPLATE/enhancement.md ‎.github/ISSUE_TEMPLATE/mds_enhancement.md

File renamed without changes.

‎.github/actions/build-connector-image/action.yml

+5-1
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ inputs:
1919
connector-name:
2020
required: true
2121
description: "EDC Connector Name in launchers/connectors/{connector-name}"
22+
deployment-type:
23+
required: true
24+
description: "Type of deployment: 'connector' or 'broker'"
2225
title:
2326
required: true
2427
description: "Docker Image Title"
@@ -62,7 +65,7 @@ runs:
6265
type=raw,value=latest,enable={{is_default_branch}}
6366
type=raw,value=release,enable=${{ startsWith(github.ref, 'refs/tags/') }}
6467
- name: "Docker: Build and Push"
65-
uses: docker/build-push-action@v4
68+
uses: docker/build-push-action@v5
6669
with:
6770
file: launchers/Dockerfile
6871
context: .
@@ -71,5 +74,6 @@ runs:
7174
labels: ${{ steps.meta.outputs.labels }}
7275
build-args: |
7376
CONNECTOR_NAME=${{ inputs.connector-name }}
77+
CONNECTOR_TYPE=${{ inputs.deployment-type }}
7478
"EDC_LAST_COMMIT_INFO_ARG=${{ env.LAST_COMMIT_INFO }}"
7579
EDC_BUILD_DATE_ARG=${{ env.BUILD_DATE }}

‎.github/workflows/ci.yml

+82
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ jobs:
6565
image-base-name: ${{ env.IMAGE_BASE_NAME }}
6666
image-name: "edc-dev"
6767
connector-name: "sovity-dev"
68+
deployment-type: "connector"
6869
title: "sovity Dev EDC Connector"
6970
description: "Extended EDC Connector built by sovity. This dev version contains no dataspace auth and can be used to quickly start a locally running EDC + EDC UI."
7071
- name: "Docker Image: edc-ce"
@@ -76,6 +77,7 @@ jobs:
7677
image-base-name: ${{ env.IMAGE_BASE_NAME }}
7778
image-name: "edc-ce"
7879
connector-name: "sovity-ce"
80+
deployment-type: "connector"
7981
title: "sovity Community Edition EDC Connector"
8082
description: "EDC Connector built by sovity. Contains sovity's Community Edition EDC extensions and requires dataspace credentials to join an existing dataspace."
8183
- name: "Docker Image: edc-ce-mds"
@@ -87,6 +89,7 @@ jobs:
8789
image-base-name: ${{ env.IMAGE_BASE_NAME }}
8890
image-name: "edc-ce-mds"
8991
connector-name: "mds-ce"
92+
deployment-type: "connector"
9093
title: "MDS Community Edition EDC Connector"
9194
description: "EDC Connector built by sovity and configured for compatibility with the Mobility Data Space (MDS). This EDC requires dataspace credentials, and additional MDS Services such as a Clearing House."
9295
- name: "Docker Image: test-backend"
@@ -98,8 +101,33 @@ jobs:
98101
image-base-name: ${{ env.IMAGE_BASE_NAME }}
99102
image-name: "test-backend"
100103
connector-name: "test-backend"
104+
deployment-type: "connector"
101105
title: "Test Data Source / Data Sink"
102106
description: "Provides a minimal data source / data sink for E2E tests."
107+
- name: "Docker Image: broker-server-dev"
108+
uses: ./.github/actions/build-connector-image
109+
with:
110+
registry-url: ${{ env.REGISTRY_URL }}
111+
registry-user: ${{ env.REGISTRY_USER }}
112+
registry-password: ${{ secrets.GITHUB_TOKEN }}
113+
image-base-name: ${{ env.IMAGE_BASE_NAME }}
114+
image-name: "broker-server-dev"
115+
connector-name: "broker-server-dev"
116+
deployment-type: "broker"
117+
title: "Broker Server (Dev)"
118+
description: "sovity EDC Broker Server. This dev version contains no auth and can be used to quickly start a locally running Broker Server + Broker UI."
119+
- name: "Docker Image: broker-server-ce"
120+
uses: ./.github/actions/build-connector-image
121+
with:
122+
registry-url: ${{ env.REGISTRY_URL }}
123+
registry-user: ${{ env.REGISTRY_USER }}
124+
registry-password: ${{ secrets.GITHUB_TOKEN }}
125+
image-base-name: ${{ env.IMAGE_BASE_NAME }}
126+
image-name: "broker-server-ce"
127+
connector-name: "broker-server-ce"
128+
deployment-type: "broker"
129+
title: "Broker Server (Community Edition)"
130+
description: "sovity EDC Broker Server. Requires dataspace credentials to join an existing dataspace."
103131
ts-api-client-library:
104132
name: TS API Client Library
105133
runs-on: ubuntu-latest
@@ -153,6 +181,60 @@ jobs:
153181
env:
154182
NODE_USER: richardtreier-sovity
155183
NODE_AUTH_TOKEN: ${{ secrets.SOVITY_EDC_CLIENT_NPM_AUTH }}
184+
ts-broker-api-client-library:
185+
runs-on: ubuntu-latest
186+
timeout-minutes: 30
187+
permissions:
188+
contents: read
189+
packages: write
190+
191+
steps:
192+
- uses: FranzDiebold/github-env-vars-action@v2
193+
- uses: actions/checkout@v4
194+
- uses: actions/setup-java@v3
195+
with:
196+
distribution: 'temurin'
197+
java-version: '17'
198+
cache: 'gradle'
199+
- uses: actions/setup-node@v4
200+
with:
201+
node-version: 16
202+
cache: 'npm'
203+
registry-url: 'https://registry.npmjs.org'
204+
cache-dependency-path: ./extensions/broker-server-api/client-ts/package-lock.json
205+
- name: Generate openapi.yaml & Client Code
206+
run: |
207+
./gradlew :extensions:broker-server-api:api:clean :extensions:broker-server-api:api:build -x test --no-daemon
208+
env:
209+
USERNAME: ${{ github.actor }}
210+
TOKEN: ${{ secrets.GITHUB_TOKEN }}
211+
- name: NPM Package Dist Tag & Version
212+
working-directory: ./extensions/broker-server-api/client-ts
213+
run: |
214+
if [ "${{ startsWith(github.ref, 'refs/tags/') }}" == "true" ]; then
215+
# Full Release
216+
VERSION="${GITHUB_REF#refs/tags/v}"
217+
DIST_TAG=latest
218+
else
219+
VERSION="0.$(date '+%Y%m%d.%H%M%S')-main-$CI_SHA_SHORT"
220+
DIST_TAG=main
221+
fi
222+
npm version $VERSION
223+
echo "DIST_TAG=$DIST_TAG" >> $GITHUB_ENV
224+
- name: Build NPM Library
225+
working-directory: ./extensions/broker-server-api/client-ts
226+
run: |
227+
npm ci && npm run build
228+
- name: Publish NPM Library
229+
if: github.ref == 'refs/heads/main' || github.ref == 'refs/tags/v*'
230+
working-directory: ./extensions/broker-server-api/client-ts
231+
run: |
232+
npm set //registry.npmjs.org/:_authToken $NODE_AUTH_TOKEN
233+
npm set //registry.npmjs.org/:username $NODE_USER
234+
npm publish --access public --tag "${{ env.DIST_TAG }}"
235+
env:
236+
NODE_USER: richardtreier-sovity
237+
NODE_AUTH_TOKEN: ${{ secrets.SOVITY_BROKER_SERVER_CLIENT_NPM_AUTH }}
156238
markdown-link-checks:
157239
name: Markdown Link Checks
158240
runs-on: ubuntu-latest

‎.github/workflows/code_analysis.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
spotbugs_active: ${{ steps.check_spotbugs.outputs.spotbugs_active }}
1818
steps:
1919
- name: Checkout code
20-
uses: actions/checkout@v3
20+
uses: actions/checkout@v4
2121
- name: Check file existence
2222
id: check_files
2323
uses: andstor/file-existence-action@v2
@@ -34,7 +34,7 @@ jobs:
3434
if: needs.is_java_project.outputs.pom_exists == 'true' && needs.is_java_project.outputs.checkstyle_active == 'true'
3535
runs-on: ubuntu-latest
3636
steps:
37-
- uses: actions/checkout@v3
37+
- uses: actions/checkout@v4
3838
- name: Set up JDK ${{ matrix.java }}
3939
uses: actions/setup-java@v3
4040
with:
@@ -48,7 +48,7 @@ jobs:
4848
if: needs.is_java_project.outputs.pom_exists == 'true' && needs.is_java_project.outputs.spotbugs_active == 'true'
4949
runs-on: ubuntu-latest
5050
steps:
51-
- uses: actions/checkout@v3
51+
- uses: actions/checkout@v4
5252
- name: Set up JDK ${{ matrix.java }}
5353
uses: actions/setup-java@v3
5454
with:

‎.github/workflows/license_scan.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,11 @@ jobs:
2828
- name: Checkout code
2929
uses: actions/checkout@v3
3030
- name: npm install (typescript-client)
31-
run: cd extensions/wrapper/clients/typescript-client && npm install
31+
run: cd extensions/wrapper/clients/typescript-client && npm clean-install
3232
- name: npm install (typescript-client-example)
33-
run: cd extensions/wrapper/clients/typescript-client-example && npm install
33+
run: cd extensions/wrapper/clients/typescript-client-example && npm clean-install
34+
- name: npm install (client-ts)
35+
run: cd extensions/broker-server-api/client-ts && npm clean-install
3436
- name: Run license scanner
3537
uses: aquasecurity/trivy-action@master
3638
with:

‎.github/workflows/secret_scan.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
fail-fast: false
1616
steps:
1717
- name: Checkout code
18-
uses: actions/checkout@v3
18+
uses: actions/checkout@v4
1919
- name: Run vulnerability scanner
2020
uses: aquasecurity/trivy-action@master
2121
with:

‎.gitignore

+2-2
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,8 @@ build
3939

4040
**/.env
4141
!.env
42-
!launchers/.env
43-
!connector/.env
42+
!launchers/.env.connector
43+
!launchers/.env.broker
4444

4545
# Log files
4646
*.log

‎CHANGELOG.md

+9-1
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,17 @@ please see [changelog_updates.md](docs/dev/changelog_updates.md).
77

88
### Overview
99

10+
Starting from version `8`, the Broker has been merged with the Community edition.
11+
12+
[The former changelog](https://github.com/sovity/edc-broker-server-extension/blob/main/CHANGELOG.md) for the Broker is still available but will not be updated anymore.
13+
14+
The Broker's version therefore jumps from version 4 to version 8.
15+
16+
The functionalities of each part, Broker and Extensions, on this release, is the same as before the change.
17+
1018
### EDC UI
1119

12-
### EDC Extensions
20+
### EDC Extensions and Broker
1321

1422
#### Major Changes
1523

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Check out our [Getting Started Section](#getting-started) on how to run a local
6666
Our sovity Community Edition EDC takes available Open Source EDC Extensions and combines them with our own
6767
open source EDC Extensions from this repository to build ready-to-use EDC Docker Images.
6868

69-
See [here](launchers/README.md) for a list of our sovity Community Edition EDC Docker Images.
69+
See [here](launchers/README.md) for a list of our sovity Community Edition EDC Docker images.
7070

7171
<p align="right">(<a href="#readme-top">back to top</a>)</p>
7272

‎SECURITY.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,5 @@ This information will help us triage your report more quickly.
2929
We prefer all communications to be in English.
3030

3131
## Attribution
32-
This file is adapted from [eclipse-edc](https://github.com/eclipse-edc/DataDashboard) project.
32+
33+
This file is adapted from [eclipse-edc/DataDashboard](https://github.com/eclipse-edc/DataDashboard) project.

‎SUMMARY.md

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Summary
2+
3+
* [Start](./README.md)
4+
* [Connector Versions](./launchers/README.md)
5+
* [Changelog](./CHANGELOG.md)
6+
7+
8+
## User Documentation
9+
10+
* [Getting Started](./docs/getting-started/README.md)
11+
* [Data Transfer Modes](./docs/getting-started/documentation/data-transfer-methods.md)
12+
* [API Wrapper](./docs/getting-started/documentation/api_wrapper.md)
13+
* [OAuth Data Address](./docs/getting-started/documentation/oauth-data-address.md)
14+
* [Parameterized Assets via UI](./docs/getting-started/documentation/parameterized_assets_via_ui.md)
15+
* [Parameterized Assets via Managment API](./docs/getting-started/documentation/parameterized_assets.md)
16+
* [Pull Data Transfer](./docs/getting-started/documentation/pull-data-transfer.md)
17+
18+
## Deployment Documentation
19+
* [Deployment Goal: Local Demo](./docs/deployment-guide/goals/local-demo)
20+
* [Deployment Goal: Development](./docs/deployment-guide/goals/development)
21+
* [Deployment Goal: Production](./docs/deployment-guide/goals/production)
22+
* [Productive Deployment Guide](./docs/deployment-guide/goals/production)
23+
* [Productive Deployment Guide 4.2.0 / MS8 / MDS 1.2](docs/deployment-guide/goals/production/4.2.0/README.md)
24+
25+
## Developer Documentation
26+
27+
* [Code of Conduct](./CODE_OF_CONDUCT.md)
28+
* [Contribution Guide](./CONTRIBUTING.md)
29+
* [Code-Style Guide](./STYLEGUIDE.md)
30+
* [Security Guide](./SECURITY.md)
31+
32+
33+
## Extensions
34+
35+
* [API Wrapper](./extensions/wrapper/README.md)
36+
* [Community Edition API](./extensions/wrapper/wrapper-api/README.md)
37+
* [Enterprise Edition API](./extensions/wrapper/wrapper-ee-api/README.md)
38+
* [Java API Client Library](./extensions/wrapper/clients/java-client/README.md)
39+
* [Java API Client Library Example](./extensions/wrapper/clients/java-client-example/README.md)
40+
* [TypeScript API Client Library](./extensions/wrapper/clients/typescript-client/README.md)
41+
* [TypeScript API Client Library Example](./extensions/wrapper/clients/typescript-client-example/README.md)
42+
* Policies
43+
* [Always True](./extensions/policy-always-true/README.md)
44+
* [Referring Connector](./extensions/policy-referring-connector/README.md)
45+
* [Time Interval](./extensions/policy-time-interval/README.md)
46+
* [Database Migration](./extensions/postgres-flyway/README.md)
47+
* [EDC UI Config](./extensions/edc-ui-config/README.md)
48+
* [Last Commit Info](./extensions/last-commit-info/README.md)

0 commit comments

Comments
 (0)