-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds support for uploading threat intelligence in Custom Format … #1493
Adds support for uploading threat intelligence in Custom Format … #1493
Conversation
…JSON (opensearch-project#1455) * revert common utils dep change Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * adds jsonpath deps Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * adds custom JsonSchema request model Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * adds parsing iocs via new tif source type - custom schema ioc upload Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * change Ioc Type variable from enum to string to support custom ioc types Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * remove ioc type check to allow custom types Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * add name and id field parsing via json path annotation Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * adds custom schema json parsing codec that parses based on JsonPath notations Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * fix stix ioc parsing with null checks on each text field Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * remove ioc type lower case conversion in ioc scan service Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * compute ioc types from iocs list instead of fetching from request Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * compute ioc types from parsed iocs in S3 threat intel source download Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * add null check Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * rewrite amazon s3 connector to compute correct input codec Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * revert if else flip for create connector Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * add logging to trace threat intel monitor execution Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * remove source type CUSTOM_SCHEMA_IOC_UPLOAD and merge the new source object into IOC_UPlOAD Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * validate that threat intel ioc type and schema that json is valid and also a legal string Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * remove iskey Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * update java docs Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * handle numbers in ioc value column Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * fix review comments Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * fix doc level monitor input constructor Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * udpate jar Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * revert build.gradle change Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> * upgrade json smart to 2.5.2 to deal with CVE-2024-57699 Signed-off-by: Surya Sashank Nistala <snistala@amazon.com> --------- Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
@@ -204,26 +199,27 @@ public static STIX2IOC parse(XContentParser xcp, String id, Long version) throws | |||
|
|||
switch (fieldName) { | |||
case NAME_FIELD: | |||
if (xcp.currentToken() == XContentParser.Token.VALUE_NULL) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious. I noticed that the STIX2IOCDto class is using a getString
helper function when parsing these fields. Could you clarify why we don't use something similar here?
private InputCodec getInputCodec(FeedConfiguration feedConfiguration, SATIFSourceConfig satifSourceConfig) { | ||
final InputCodec inputCodec; | ||
if (satifSourceConfig != null && satifSourceConfig.getIocSchema() != null) { | ||
logger.info("Parsing custom schema JSON from S3 for threat intel source [{}]", satifSourceConfig.getName()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nitpick: Would using the source config ID make more sense for this log? Or do we have validation in place to prevent customers from creating configs with the same name?
Approved as PR to 2.x was reviewed and merged. |
The security test workflow should be fixed now. You'll need to rebase this commit |
Cherry-picks from 2.x (#1455) to main
Closed original PR #1450 to main due to dep conflicts in CIs