Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HuggingFace download broken on Julia 1.10 (by ValSplit.jl?) #147

Closed
svilupp opened this issue Aug 8, 2023 · 1 comment
Closed

HuggingFace download broken on Julia 1.10 (by ValSplit.jl?) #147

svilupp opened this issue Aug 8, 2023 · 1 comment

Comments

@svilupp
Copy link

svilupp commented Aug 8, 2023

First of all, thank you for the awesome package!

If I run a snippet from the docs to load bert config from HF Hub (load_config("bert-base-cased")), I get an error ERROR: MethodError: no method matching iterate(::Nothing)

Possible root cause: I validated that HF Hub, download and parser are working correctly. So I believe ValSplit.jl is the culprit on this line

It works on Julia 1.9 as expected, but not 1.10. It's a shame because transformers loads so much faster on 1.10!

MWE on Julia 1.10

using Transformers
using Transformers.HuggingFace

load_config("bert-base-cased")
# ERROR: MethodError: no method matching iterate(::Nothing)

Full stacktrace

ERROR: MethodError: no method matching iterate(::Nothing)

Closest candidates are:
  iterate(::Base.RegexMatchIterator)
   @ Base regex.jl:686
  iterate(::Base.RegexMatchIterator, ::Any)
   @ Base regex.jl:686
  iterate(::LLVM.ValueUseSet)
   @ LLVM ~/.julia/packages/LLVM/Od0DH/src/core/value.jl:146
  ...

Stacktrace:
  [1] iterate(::Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing})
    @ Base.Iterators ./iterators.jl:516
  [2] iterate(::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"})
    @ Base ./generator.jl:44
  [3] _collect(::Type{Method}, itr::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"}, isz::Base.SizeUnknown)
    @ Base ./array.jl:698
  [4] collect(::Type{Method}, itr::Base.Generator{Base.Iterators.Filter{Tricks.var"#7#9"{Nothing}, Nothing}, Tricks.var"#6#8"})
    @ Base ./array.jl:692
  [5] _methods(f_type::Any, t_type::Any, mod::Nothing)
    @ Tricks ~/.julia/packages/Tricks/7oAyo/src/Tricks.jl:146
  [6] _methods(f_type::Any, t_type::Any, mod::Nothing)
    @ Tricks ~/.julia/packages/Tricks/7oAyo/src/Tricks.jl:143 [inlined]
  [7] #s3#3
    @ ValSplit ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:42 [inlined]
  [8] var"#s3#3"(T::Any, N::Any, P::Any, ::Any, f::Any, types::Any, idx::Any, ptype::Any)
    @ ValSplit ./none:0
  [9] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
    @ Core ./boot.jl:600
 [10] valarg_params(f::typeof(config_type), types::Type{Tuple{Val}}, idx::Val{1}, ptype::Type{Symbol})
    @ ValSplit ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:60 [inlined]
 [11] config_type(model_type::Symbol)
    @ Transformers.HuggingFace ~/.julia/packages/ValSplit/MMCz3/src/ValSplit.jl:141
 [12] load_config(model_type::Symbol, cfg::JSON3.Object{Vector{UInt8}, Vector{UInt64}})
    @ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:67
 [13] _load_config(cfg::JSON3.Object{Vector{UInt8}, Vector{UInt64}})
    @ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:56
 [14] load_config(model::String; kw::@Kwargs{})
    @ Transformers.HuggingFace ~/.julia/packages/Transformers/qhUJm/src/huggingface/configs/auto.jl:42
 [15] top-level scope
    @ ~/Documents/Julia-training-range/transformers/first.jl:12
    ```
    
Version: Transformers 0.2.7 (latest)

Versioninfo:
> Julia Version 1.10.0-beta1
> Commit 6616549950e (2023-07-25 17:43 UTC)
> Platform Info:
>   OS: macOS (arm64-apple-darwin22.4.0)
>   CPU: 8 × Apple M1 Pro
>   WORD_SIZE: 64
>   LIBM: libopenlibm
>   LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
>   Threads: 1 on 6 virtual cores
> Environment:
>   JULIA_EDITOR = code
@chengchingwen
Copy link
Owner

It seems to be caused by ValSplit. We might need to wait until that get fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants