Detect outliers using LOF

[urschrei]

• I agree to follow the project's code of conduct.
• I added an entry to CHANGES.md if knowledge of this change could be valuable to users.

---

Still draft due to:

• perf: two of the LRD passes look very similar. I need to figure out whether this can be tidied up, or whether it's ultimately not going to affect perf much Can't think of a particularly more elegant way to do this
• Tests, tests, tests
• Other geometries / collections?

[urschrei]

I suspect there's a way to calculate LRDs for a subset (the point's neighbours) as we go, but I can't get it right so we're stuck with local_reachability_densities and a redundant kth_dist calculation for now. I think it would ultimately end up being two passes over the data anyway, but it might use less space.

[urschrei]

The paper notes that in order to calculate outliers in a robust fashion it might be necessary to calculate an "ensemble" using a range of nearest-neighour values. Since the r*-tree is expensive to fill and can be re-used, it would make sense to have a "prepared outlier calculator" API that holds on to the tree, allowing you to pass in a new k_neighbours value and getting a re-calculated dataset back out – a dataset is guaranteed to have the same dimensions (one LofScore entry per input point), so it will be easy to either aggregate scores once you've run it over a range of input neighbour sizes, or calculate them as you go in order to save space if you know what aggregate measure will give you good results.

All suggestions for an API gratefully received.

[urschrei]

The paper notes that in order to calculate outliers in a robust fashion it might be necessary to calculate an "ensemble" using a range of nearest-neighour values. Since the r*-tree is expensive to fill and can be re-used, it would make sense to have a "prepared outlier calculator" API that holds on to the tree, allowing you to pass in a new k_neighbours value and getting a re-calculated dataset back out – a dataset is guaranteed to have the same dimensions (one LofScore entry per input point), so it will be easy to aggregate scores once you've run it over a range of input neighbour sizes

This is done!

[michaelkirk]

Nice work! I have some questions, but all my suggestions were minor and should be considered "take it or leave it".

[urschrei]

All review comments addressed, I think.

[michaelkirk]

LGTM! Not sure if you were looking for more reviews, so I'll leave it to you if you want to merge it.

[urschrei]

Thanks for the review! I think it's in good shape now but I'll leave it open for a couple of days / until @frewsxcv has kicked the tires on it.

[urschrei]

bors r+

[bors]

Build succeeded:

• ci result