Add feature processor to convert geojson feature to geoshape field #15

VijayanB · 2022-01-07T02:43:03Z

Description

Added new Processor of type geojson-feature to intercept GeoJSON Feature and convert into typical OpenSearch document.
Example: A GeoJSON object of type “Feature”, contains geometry object of type “LineString”.

{
    "type": "Feature",
    "geometry": {
        "type": "LineString",
        "coordinates": [
            [102.0, 0.0],
            [103.0, 1.0],
            [104.0, 0.0],
            [105.0, 1.0]
        ]
    },
    "properties": {
        "key0": "value0",
        "key1": "value1",
        "key2": "value2"
    }
}

The above feature can be converted to typical document for an index which has "location" as type geo_shape, by FeatureProcessor.

{
    "location": {
        "type": "LineString",
        "coordinates": [
            [102.0, 0.0],
            [103.0, 1.0],
            [104.0, 0.0],
            [105.0, 1.0]
        ]
    },
     "key0": "value0",
     "key1": "value1",
     "key2": "value2"
}

This processor converts Geometry Object into geo_shape field value, removes GeoJSON metadata like type, and convert properties into direct fields.

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

jmazanec15

Can you add more detail to the description?

src/main/java/org/opensearch/geospatial/geojson/Feature.java

jmazanec15 · 2022-01-18T17:22:30Z

src/main/java/org/opensearch/geospatial/geojson/Feature.java

+public class Feature {
+    public static final String TYPE = "Feature";
+    private final Map<String, Object> geometry;
+    private Map<String, Object> properties = new HashMap<>();


Why create this object here?

Moved it to constructor

jmazanec15 · 2022-01-18T17:23:06Z

src/main/java/org/opensearch/geospatial/geojson/GeoJSONFactory.java

+
+import java.util.Map;
+
+public class GeoJSONFactory {


Should this be FeatureFactory if it produces a Feature?

jmazanec15 · 2022-01-18T17:24:31Z

src/main/java/org/opensearch/geospatial/geojson/GeoJSONFactory.java

+    }
+
+    private static Feature readFeature(Map<String, Object> input) {
+        Map<String, Object> geometryMap = (Map<String, Object>) input.get(GEOJSON_GEOMETRY_KEY);


Does this need to be validated?

jmazanec15 · 2022-01-18T17:27:49Z

src/main/java/org/opensearch/geospatial/processor/GeoJSONFeatureProcessor.java

+
+    @Override
+    public IngestDocument execute(IngestDocument ingestDocument) {
+        Feature feature = GeoJSONFactory.create(ingestDocument.getSourceAndMetadata());


Can you add comments here explaining what is being done?

martin-gaievski · 2022-01-18T19:37:50Z

src/main/java/org/opensearch/geospatial/geojson/Feature.java

+     */
+    static class FeatureBuilder {
+
+        private Feature feature;


can we make it final?

martin-gaievski · 2022-01-18T19:41:35Z

src/main/java/org/opensearch/geospatial/geojson/FeatureFactory.java

+     * @return Feature Instance from input
+     * @throws IllegalArgumentException if invalid input is provided
+     */
+    public static Feature create(Map<String, Object> input) throws IllegalArgumentException {


Why IllegalArgumentException is defined in the method signature, it's unchecked exception?

martin-gaievski · 2022-01-18T19:48:04Z

src/main/java/org/opensearch/geospatial/geojson/FeatureFactory.java

+        if (!(geometry instanceof Map)) {
+            throw new IllegalArgumentException("key: " + Feature.GEOMETRY_KEY + "is not an instance of type Map");
+        }
+        Map<String, Object> geometryMap = (Map<String, Object>) geometry;


nit: are we sure this is map of <String, Object>? Although I'm sure how to check that properly, effectively you're getting Map<Object,Object>

created new method to convert

martin-gaievski

Lgtm, thank you

jmazanec15 · 2022-01-18T23:15:57Z

src/main/java/org/opensearch/geospatial/geojson/Feature.java

+     *
+     * @return Feature's type value
+     */
+    public String getType() {


Why do we need a getter on a public static type?

Good catch. I forgot to remove this method after i made it as public static constant.

jmazanec15 · 2022-01-18T23:18:13Z

src/main/java/org/opensearch/geospatial/geojson/FeatureFactory.java

+        Map<String, Object> geometryMap = toStringObjectMap(geometry);
+        FeatureBuilder featureBuilder = new FeatureBuilder(geometryMap);
+        Object properties = input.get(Feature.PROPERTIES_KEY);
+        if (properties == null)


nit: Why not wrap this in curly braces?

jmazanec15 · 2022-01-18T23:18:45Z

src/main/java/org/opensearch/geospatial/geojson/FeatureFactory.java

+        if (properties == null)
+            return featureBuilder;
+
+        if (!(geometry instanceof Map)) {


I think this is supposed to be properties

copy paste from previous check :( I corrected it now.

Added new Processor of type geojson-feature. Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

VijayanB requested a review from a team January 7, 2022 02:43

VijayanB changed the title ~~Add processor~~ Add GeoJSONFeature Processor Jan 7, 2022

VijayanB force-pushed the add-processor branch 2 times, most recently from f89d8e7 to 415380e Compare January 7, 2022 02:44

VijayanB self-assigned this Jan 7, 2022

VijayanB force-pushed the add-processor branch 14 times, most recently from 796a1a1 to 5de6691 Compare January 15, 2022 06:55

VijayanB force-pushed the add-processor branch from 5de6691 to 027dc71 Compare January 17, 2022 22:31

jmazanec15 reviewed Jan 18, 2022

View reviewed changes

VijayanB force-pushed the add-processor branch 5 times, most recently from 032d441 to a0727b7 Compare January 18, 2022 19:45

martin-gaievski reviewed Jan 18, 2022

View reviewed changes

VijayanB force-pushed the add-processor branch 3 times, most recently from 0ee11a5 to 273a13d Compare January 18, 2022 21:26

VijayanB force-pushed the add-processor branch 2 times, most recently from 70b9b6a to c2b99b8 Compare January 18, 2022 21:39

VijayanB changed the title ~~Add GeoJSONFeature Processor~~ Add Feature Processor to convert geojson feature to geoshape field Jan 18, 2022

VijayanB changed the title ~~Add Feature Processor to convert geojson feature to geoshape field~~ Add feature processor to convert geojson feature to geoshape field Jan 18, 2022

VijayanB force-pushed the add-processor branch 2 times, most recently from 5c808f8 to 54c7553 Compare January 18, 2022 21:53

VijayanB requested review from martin-gaievski and jmazanec15 January 18, 2022 21:54

martin-gaievski approved these changes Jan 18, 2022

View reviewed changes

jmazanec15 reviewed Jan 18, 2022

View reviewed changes

VijayanB added 2 commits January 18, 2022 15:51

Add GeoJSONFeature Processor

ec5f0c3

Added new Processor of type geojson-feature. Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

Update license headers

84eb6a0

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

VijayanB force-pushed the add-processor branch from 54c7553 to 84eb6a0 Compare January 18, 2022 23:52

VijayanB requested a review from jmazanec15 January 18, 2022 23:54

jmazanec15 approved these changes Jan 18, 2022

View reviewed changes

VijayanB merged commit c9efedb into opensearch-project:main Jan 18, 2022

VijayanB deleted the add-processor branch January 18, 2022 23:59

junqiu-lei added v2.2.0 feature labels Aug 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add feature processor to convert geojson feature to geoshape field #15

Add feature processor to convert geojson feature to geoshape field #15

VijayanB commented Jan 7, 2022 •

edited

Loading

jmazanec15 left a comment

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

martin-gaievski Jan 18, 2022

VijayanB Jan 18, 2022

martin-gaievski Jan 18, 2022

VijayanB Jan 18, 2022

martin-gaievski Jan 18, 2022

VijayanB Jan 18, 2022

martin-gaievski left a comment

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

jmazanec15 Jan 18, 2022

VijayanB Jan 18, 2022

Add feature processor to convert geojson feature to geoshape field #15

Add feature processor to convert geojson feature to geoshape field #15

Conversation

VijayanB commented Jan 7, 2022 • edited Loading

Description

Check List

jmazanec15 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martin-gaievski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VijayanB commented Jan 7, 2022 •

edited

Loading