Skip to content

Commit 61023b6

Browse files
bedrogebedroge
authored
Merge pull request #2 from trz42/dev-env-instructions
Added development setup instructions to README.md
2 parents 7bc4517 + 114f7b3 commit 61023b6

File tree

1 file changed

+186
-0
lines changed

1 file changed

+186
-0
lines changed

‎README.md

+186
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,189 @@ Requires:
1818
```
1919
pip3 install --user -r requirements.txt
2020
```
21+
22+
# Detailed instructions to set up development environment
23+
24+
The following sections describe and illustrate the steps necessary to set up the development environment for the EESSI bot.
25+
26+
## Prerequisites
27+
28+
- GitHub account
29+
- GitHub repository on whose events the bot shall react to
30+
- Linux machine where the EESSI bot shall run (some steps may require sudo access)
31+
32+
## Step 1: Smee.io channel and smee client
33+
34+
We use smee.io as a service to relay events from GitHub to the EESSI bot. To do so, create a new channel on the page https://smee.io and note the URL, e.g., https://smee.io/CHANNEL_ID
35+
36+
On the Linux machine which runs the EESSI bot we need a tool which receives events relayed from https://smee.io/CHANNEL_ID and forwards it to the EESSI bot. We use the Smee client for this. The Smee client can be installed globally with
37+
38+
```
39+
npm install -g smee-client
40+
```
41+
42+
or per user
43+
44+
```
45+
npm install smee-client
46+
```
47+
48+
If you don't have `npm` on your system and don't have sudo access to easily install it, you may use a container as follows
49+
50+
```
51+
mkdir smee
52+
cd smee
53+
singularity pull docker://node
54+
singularity exec node_latest.sif npm install smee-client
55+
cat << EOF > smee
56+
#!/usr/bin/env bash
57+
58+
BASEDIR=$(dirname "$0")
59+
60+
singularity exec $BASEDIR/node_latest.sif $BASEDIR/node_modules/smee-client/bin/smee.js "$@"
61+
EOF
62+
63+
chmod 700 smee
64+
export PATH=$PATH:$PWD
65+
```
66+
67+
Finally, run the Smee client as follows
68+
69+
```
70+
smee --url https://smee.io/CHANNEL_ID
71+
```
72+
73+
Alternatively, you may use a container providing the smee client. For example,
74+
75+
```
76+
singularity pull docker://deltaprojects/smee-client
77+
singularity run smee-client_latest.sif --url https://smee.io/CHANNEL_ID
78+
```
79+
80+
## <a name="step2"></a>Step 2: Registering GitHub App
81+
82+
We first need to register a GitHub App, link it to the Smee.io channel, set a secret token to verify the webhook sender, set some permissions for the app, subscribe it to selected events and define that this app should only be installed in your account.
83+
84+
At the [app settings page](https://github.com/settings/apps) click "New GitHub App" and fill in the page, particular the following fields
85+
- GitHub App name: give the app a name of you choice
86+
- Homepage URL: use the Smee.io channel (https://smee.io/CHANNEL_ID) created in Step 1
87+
- Webhook URL: use the Smee.io channel (https://smee.io/CHANNEL_ID) created in Step 1
88+
- Webhook secret: create a secret token which is used to verify the webhook sender
89+
- Permissions: assign permissions to the app it needs (e.g., read access to commits, issues, pull requests); those can be changed later on; some permissions (e.g., metadata) will be selected automatically because of others you have chosen
90+
- Events: subscribe the app to events it shall react on (e.g., related to pull requests)
91+
- Select that the app can only be installed by this (your) GitHub account
92+
93+
Click on "Create GitHub App"
94+
95+
## Step 3: Installing GitHub App (might trigger first event to EESSI bot)
96+
97+
You need to install the GitHub App -- essentially telling GitHub to link the app to an account and one, several or all repositories on whose events the app then should act upon.
98+
99+
Go to the page https://github.com/settings/apps and select the app you want to install by clicking on the icon left to the apps name or on the "Edit" button right to the name of the app. On the next page you should see the menu item "Install App" on the left-hand side. When you click on this you should see a page with a list of accounts you can install the app on. Choose one and click on the "Install" button next to it. This leads to a page where you can select the repositories on whose the app should react to. Select one, multiple or all and click on the "Install" button.
100+
101+
## Step 4: Installing the EESSI bot on Linux machine
102+
103+
The EESSI bot for the software layer is available from https://github.com/EESSI/eessi-bot-software-layer
104+
105+
Get the EESSI bot onto the Linux machine by running something like
106+
107+
```
108+
git clone https://github.com/EESSI/eessi-bot-software-layer.git
109+
```
110+
111+
If you want to develop the EESSI bot, it is recommended that you fork the repository and use the fork on the Linux machine.
112+
113+
The EESSI bot requires some Python packages to be installed. See the top of this page, or simply run (the `requirements.txt` file is provided by the EESSI bot repository)
114+
```
115+
pip3 install --user -r requirements.txt
116+
```
117+
118+
**Troubles installing some of the requirements?**
119+
You may try to install some of the dependencies by fixing their version. For example, on the CitC cluster (https://github.com/EESSI/hackathons/tree/main/2021-12/citc) installing PyGithub failed due to some problem installing its dependency PyNaCl. Apparently, PyGithub only required version 1.4.0 of PyNaCl but the most recent version 1.5.0 failed to install. Hence, when installing PyNaCl version 1.4.0 first, then PyGithub could be installed. Example commands
120+
121+
```
122+
pip3 install --user PyNaCl==1.4.0
123+
pip3 install --user -r requirements.txt
124+
```
125+
126+
### Step 4.1 Using the development version of PyGHee
127+
128+
The above command `pip3 install --user -r requirements.txt` installs the latest release of the PyGHee library. If you want to use the development version, i.e., what is available from GitHub or your own local copy, you have to set `PYTHONPATH` correctly. Assume the library's main directory is `SOME_PATH/PyGHee` then do the following in the terminal/shell/script where you run the bot:
129+
130+
```
131+
export PYTHONPATH=SOME_PATH/PyGHee
132+
```
133+
134+
## Step 5: Configuring and running EESSI bot on Linux machine
135+
136+
You need to set up two environment variables: `GITHUB_TOKEN` and `GITHUB_APP_SECRET_TOKEN`.
137+
138+
### Step 5.1: GitHub Personal Access Token (PAT)
139+
140+
Create a Personal Access Token (PAT) for your GitHub account via the page https://github.com/settings/tokens where you find a button "Generate new token".
141+
Give it meaningful name (field titled "Note") and set the expiration date. Then select the scopes this PAT will be used for. Then click "Generate token". On the result page, take note/copy the resulting token string -- it will only be shown once.
142+
143+
On the Linux machine set the environment variable `GITHUB_TOKEN`, e.g.
144+
```
145+
export GITHUB_TOKEN='THE_TOKEN_STRING'
146+
```
147+
148+
### Step 5.2: GitHub App Secret Token
149+
The GitHub App Secret Token is used to verify the webhook sender. You should have created one already when registering a new GitHub App in Step 1.
150+
151+
On the Linux machine set the environment variable `GITHUB_APP_SECTRET_TOKEN`, e.g.
152+
```
153+
export GITHUB_APP_SECRET_TOKEN='THE_SECRET_TOKEN_STRING'
154+
```
155+
Note, depending on the characters used in the string you will likely have to use single quotes when setting the value of the environment variable.
156+
157+
### Step 5.3: Create a private key and store it on the Linux machine
158+
The private key is needed to let the app authenticate when updating information at the repository such as commenting on PRs, adding labels, etc. You can create the key at the page of the GitHub App you have registered in Step 1.
159+
160+
Open the page https://github.com/settings/apps and then click on the icon left to the name of the GitHub App for the EESSI bot or the "Edit" button for the app. Near the end of the page you will find a section "Private keys" where you can create a private key by clicking on the button "Generate a private key". The private key should be automatically downloaded to your local computer. Copy it to the Linux machine and note the full path to it.
161+
162+
### Step 5.4: Obtain bot repository
163+
164+
The bot needs a few scripts. These and an example configuration file are provided by the repository [EESSI/eessi-bot-software-layer](https://github.com/EESSI/eessi-bot-software-layer) (or your fork of it).
165+
166+
First, clone the EESSI/eessi-bot-software-layer repository (or your fork of it) by running
167+
168+
```
169+
git clone https://github.com/EESSI/eessi-bot-software-layer.git
170+
```
171+
172+
### Step 5.5: Create the configuration file `app.cfg`
173+
174+
After cloning the bot's repository, change directory with `cd eessi-bot-software-layer` and note the full path of the directory (`pwd`).
175+
176+
If there is no `app.cfg` in the directory, create an initial version from `app.cfg.example`.
177+
178+
```
179+
cp -i app.cfg.example app.cfg
180+
```
181+
182+
Now set some values as follows:
183+
184+
```
185+
private_key = FULL_PATH_TO_PRIVATE_KEY
186+
build_job_script = PATH_TO_BOT_REPO/scripts/eessi-bot-build.slurm
187+
```
188+
189+
You will also need to set the `app_id` and the `installation_id`. You find the id of your GitHub App via the page [GitHub Apps](https://github.com/settings/apps). On this page, select the app you have registered in [Step 2](#step2). On page of the app you will find the `app_id` listed as 'App ID'. For the `installation_id` select 'Install App' in the menu on the left-hand side. Then click on the gearwheel button of the installation (to the right of the 'Installed' label). The URL of the resulting page contains the `installation_id` -- the number after the last '/'.
190+
191+
Replace default values for `app_id` and `installation_id` in `app.cfg` with the values you have obtained as described above.
192+
193+
### Step 5.6: Run the EESSI bot
194+
195+
Change directory to `eessi-bot-software-layer` (which was created by cloning the repository in Step 5.4 - either the original one from EESSI or your fork). Then, simply run the bot by executing
196+
```
197+
./run.sh
198+
```
199+
200+
Note, if you run the bot on a frontend of a cluster with multiple frontends make sure that both the Smee client and the bot run on the same machine.
201+
202+
The bot will log events into the file `pyghee.log`.
203+
204+
## Testing EESSI bot
205+
206+
The easiest test may be a change -- creating a branch, committing a change, creating a pull request, etc -- to one of the repositories you have selected when installing the app in Step 3. Which events may be forwarded depends on which events you have subscribed the app to when you registered the app in Step 2.

0 commit comments

Comments
 (0)