Code

Add spellchecking#51

Code

Draft

bminixhofer wants to merge 16 commits intomainbminixhofer/nlprule:mainfrom spellcheckbminixhofer/nlprule:spellcheckCopy head branch name to clipboard

Commits

Commits on Mar 1, 2021

• load words from spelling dictionary

  bminixhofer

  committed
  582987f

  Copy full SHA for 582987f

• add spellcheck module

  bminixhofer

  committed
  b2ea80a

  Copy full SHA for b2ea80a

Commits on Mar 2, 2021

• more clone derives, store Arc<Tokenizer> in Rules struct

  bminixhofer

  committed
  7f6458a

  Copy full SHA for 7f6458a

• treat spellchecker language variants properly

  bminixhofer

  committed
  026ffb7

  Copy full SHA for 026ffb7

Commits on Mar 5, 2021

• switch to fst based spelling correction

  bminixhofer

  committed
  a0be3c2

  Copy full SHA for a0be3c2

Commits on Mar 6, 2021

• add tests for spellint, add hashset to check if word is valid

  bminixhofer

  committed
  f086fc5

  Copy full SHA for f086fc5

• add python bindings

  bminixhofer

  committed
  0c66c78

  Copy full SHA for 0c66c78

Commits on Mar 13, 2021

• add support for multiword spelling correction, add contraction mapping

  bminixhofer

  committed
  93881da

  Copy full SHA for 93881da

Commits on Mar 15, 2021

• proper multiword spellchecker support, ignore action=ignore_spelling tokens

  bminixhofer

  committed
  16781b5

  Copy full SHA for 16781b5

• add whitelist support, add variant_checker struct for cleaner separation

  bminixhofer

  committed
  79ac926

  Copy full SHA for 79ac926

• extend distance to optimal string alignment distance

  bminixhofer

  committed
  9f15524

  Copy full SHA for 9f15524

Commits on Mar 16, 2021

• improve docs

  bminixhofer

  committed
  57c06e2

  Copy full SHA for 57c06e2

• reduce repetition, fix warnings

  bminixhofer

  committed
  0a738db

  Copy full SHA for 0a738db

• fix test, qol improvement in dev scripts

  bminixhofer

  committed
  f474d91

  Copy full SHA for f474d91

• fix doctests

  bminixhofer

  committed
  f1fdee7

  Copy full SHA for f1fdee7

• fix python ci

  bminixhofer

  committed
  b0348bb

  Copy full SHA for b0348bb