Ignore parsing hidden files on kedro viz --lite

[ravi-kumar-pilla]

Description

Resolves #2231

Development notes

• Added spinner to show users a sign of processing
• Added logic to ignore hidden files and gitignored files (Reused code of gitignore in autoreload done by @jitu5)
• Updated log level to Warning as discussed
• Updated tests

QA notes

• All tests should pass

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added new entries to the RELEASE.md file
• Added tests to cover my changes

[rashidakanchwala]

I noticed, the dataset type is now intergrations.UnavailableDataset on Kedro-Viz

[rashidakanchwala]

lgtm!

[astrojuanlu]

How can we verify that this effectively solves the issue?

[ravi-kumar-pilla]

How can we verify that this effectively solves the issue?

Hi @astrojuanlu , So the slowness might be due to many reasons but the issue's #2231 root cause is the parser parsing hidden files and gitignored files. This PR helps in resolving this issue. The test case here confirms it ignores hidden files.

If the slowness is reported again, then we can check for the cause and fix it. It would be hard to predict what might cause slowness without someone reporting bugs like #2231. Wdyt ?

[astrojuanlu]

The test case here confirms it ignores hidden files.

Good 👍🏼

If the slowness is reported again [...]

Might be wiser to tell the user to try a pip install -e "kedro-viz @ git+..." and check for themselves, just to be 100 % sure, rather than waiting for a release to happen. Just a suggestion.

Code-wise I have nothing to say but also without a reproducer I can't QA this.

[SajidAlamQB]

Thank you @ravi-kumar-pilla, this looks good to me!

[jitu5]

Nice!