Release 1.5.0 to main #327
Conversation
fix: #OBS-I354 dataset level metrics fixes and addition
fix: #OBS-I354 throw alerts on failed events greater than 0
fix: #OBS-I452 Dataset alias attach and detach functionality
fix: #OBS-I452 Dataset in/out apis updated for dataset check using alias
feat: #OBS-I452 : list api changes to return alias name
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const fieldValues = fields ? _.split(fields as string, ",") : []; | ||
| const invalidFields = _.difference(fieldValues, Object.keys(DatasetDraft.getAttributes())); | ||
| const invalidFields = mode === "edit" ? _.difference(fieldValues, Object.keys(DatasetDraft.getAttributes())) : _.difference(fieldValues, Object.keys(Dataset.getAttributes())); |
Check failure
Code scanning / CodeQL
Loop bound injection High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI about 1 month ago
To fix the problem, we need to ensure that the fields parameter is validated before it is processed. Specifically, we should check that the fields parameter is a string and that the resulting fieldValues array has a reasonable length. If the length exceeds a predefined limit, we should throw an error or handle it appropriately.
- Add a validation step to check if
fieldsis a string and if the length offieldValuesis within an acceptable range. - If the validation fails, throw an error or return an appropriate response.
| @@ -20,2 +20,5 @@ | ||
| const { fields, mode } = req.query; | ||
| if (fields && (typeof fields !== 'string' || fields.length > 1000)) { | ||
| throw obsrvError(dataset_id, "DATASET_INVALID_FIELDS", `The 'fields' parameter is invalid or too long.`, "BAD_REQUEST", 400); | ||
| } | ||
| const fieldValues = fields ? _.split(fields as string, ",") : []; |
| const fieldValues = fields ? _.split(fields as string, ",") : []; | ||
| const invalidFields = _.difference(fieldValues, Object.keys(DatasetDraft.getAttributes())); | ||
| const invalidFields = mode === "edit" ? _.difference(fieldValues, Object.keys(DatasetDraft.getAttributes())) : _.difference(fieldValues, Object.keys(Dataset.getAttributes())); |
Check failure
Code scanning / CodeQL
Loop bound injection High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI about 1 month ago
To fix the problem, we need to ensure that the fields parameter is either an array or a string that can be safely split into an array. Additionally, we should limit the size of the resulting fieldValues array to prevent excessive resource consumption.
- Check if the
fieldsparameter is a string and split it into an array. - Limit the size of the
fieldValuesarray to a reasonable maximum length. - If the
fieldsparameter is not a valid string, default to an empty array.
| @@ -20,3 +20,4 @@ | ||
| const { fields, mode } = req.query; | ||
| const fieldValues = fields ? _.split(fields as string, ",") : []; | ||
| const MAX_FIELDS_LENGTH = 100; // Define a reasonable maximum length for fieldValues | ||
| const fieldValues = (typeof fields === 'string') ? _.split(fields, ",").slice(0, MAX_FIELDS_LENGTH) : []; | ||
| const invalidFields = mode === "edit" ? _.difference(fieldValues, Object.keys(DatasetDraft.getAttributes())) : _.difference(fieldValues, Object.keys(Dataset.getAttributes())); |
| @@ -62,6 +63,7 @@ | |||
| router.get("/connectors/read/:id", setDataToRequestObject("api.connectors.read"), onRequest({entity: Entity.Management }), telemetryAuditStart({action: telemetryActions.readConnectors, operationType: OperationType.GET}), checkRBAC.handler(), ConnectorsRead); | |||
| router.post("/datasets/import", setDataToRequestObject("api.datasets.import"), onRequest({ entity: Entity.Management }), checkRBAC.handler(), DatasetImport); | |||
| router.post("/connector/register", setDataToRequestObject("api.connector.register"), onRequest({ entity: Entity.Management }), connectorRegisterController); | |||
| router.post("/datasets/alias", setDataToRequestObject("api.datasets.alias"), onRequest({ entity: Entity.Management }), checkRBAC.handler(), datasetAlias); | |||
Check failure
Code scanning / CodeQL
Missing rate limiting High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI about 1 month ago
To fix the problem, we need to introduce rate limiting to the Express application. The best way to do this is by using the express-rate-limit package, which allows us to easily set up rate limiting middleware. We will configure the rate limiter to allow a maximum of 100 requests per 15 minutes and apply it to all routes.
- Install the
express-rate-limitpackage. - Import the
express-rate-limitpackage in theRouter.tsfile. - Configure the rate limiter with the desired settings.
- Apply the rate limiter to all routes.
| @@ -1,2 +1,3 @@ | ||
| import express from "express"; | ||
| import rateLimit from "express-rate-limit"; | ||
| import dataIn from "../controllers/DataIngestion/DataIngestionController"; | ||
| @@ -38,4 +39,10 @@ | ||
|
|
||
| const limiter = rateLimit({ | ||
| windowMs: 15 * 60 * 1000, // 15 minutes | ||
| max: 100, // limit each IP to 100 requests per windowMs | ||
| }); | ||
|
|
||
| export const router = express.Router(); | ||
|
|
||
| router.use(limiter); | ||
| router.post("/data/in/:dataset_id", setDataToRequestObject("api.data.in"), onRequest({ entity: Entity.Data_in }), telemetryAuditStart({action: telemetryActions.createDataset, operationType: OperationType.CREATE}), checkRBAC.handler(), dataIn); |
| @@ -68,3 +68,4 @@ | ||
| "winston": "~2.4.3", | ||
| "winston-daily-rotate-file": "~3.2.1" | ||
| "winston-daily-rotate-file": "~3.2.1", | ||
| "express-rate-limit": "^7.5.0" | ||
| }, |
| Package | Version | Security advisories |
| express-rate-limit (npm) | 7.5.0 | None |
No description provided.