Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to import from epgshare01 #2

Closed
max1c opened this issue Dec 26, 2023 · 3 comments
Closed

Failing to import from epgshare01 #2

max1c opened this issue Dec 26, 2023 · 3 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@max1c
Copy link

max1c commented Dec 26, 2023

I'm not sure if it's because it's too large but the epg_ripper_ALL_SOURCES1.xml.gz from https://epgshare01.online/epgshare01/ is failing to import. The others ones seem to work well.

EDIT: Also, is there a portable version?

@sebdelsol
Copy link
Owner

sebdelsol commented Dec 28, 2023

I'm not sure if it's because it's too large

Yeah the xml is huge (~20 000 channels and ~2 000 000 programmes) and causes memory issues. So I'll have to find a way to serialize it on the fly. Expect a new version in the following days.

Also, is there a portable version?

There's no portable version anymore : I've switched from Nuitka single file to Nsis because it gives way less AV false positive. And maintaining a portable version became a PITA. So I ditched it.

@sebdelsol
Copy link
Owner

sebdelsol commented Dec 29, 2023

It's fixed in v1.4.12.4:
I now use lxml.etree.iterparse that processes the tree on the fly and prevents the memory issue.
The whole epg is processed into a dictionary: for this huge one it takes only 1GB instead of 10GB when it was an xml tree.
The epg is longer to process though : it takes 30s to 1mn for this one. So its progress is shown in the UI and can be interrupted.

EDIT: thanks a lot for raising this issue.

@max1c
Copy link
Author

max1c commented Dec 29, 2023

Hey, thanks so much for a quick turn around. I appreciate it. I also understand the complications with portable version. Cheers!

@sebdelsol sebdelsol added good first issue Good for newcomers enhancement New feature or request labels Jan 1, 2024
@sebdelsol sebdelsol added bug Something isn't working and removed enhancement New feature or request labels Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants