You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In Spark 3.2.1, a typo in the method signature for DataSourceUtils.creteDateRebaseFuncInWrite was fixed. apache/spark#34978 Of course, this change is neither backward nor forward compatible. Since the spark-avro jar is a compile-time dependency for ABRiS, other applications that create a fat-jar with the spark-avro v3.2.0 will experience a NoSuchMethodError when running this against Spark 3.2.1 (see below), because the spark-avro jar is usually not part of the Spark distribution.
Cause: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.DataSourceUtils$.creteDateRebaseFuncInRead(Lscala/Enumeration$Value;Ljava/lang/String;)Lscala/Function1;
at org.apache.spark.sql.avro.AvroDeserializer.<init>(AvroDeserializer.scala:67)
at org.apache.spark.sql.avro.AvroDeserializer.<init>(AvroDeserializer.scala:61)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.spark.sql.avro.AbrisAvroDeserializer$$anonfun$1.applyOrElse(AbrisAvroDeserializer.scala:40)
at org.apache.spark.sql.avro.AbrisAvroDeserializer$$anonfun$1.applyOrElse(AbrisAvroDeserializer.scala:38)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
at scala.util.Failure.recover(Try.scala:234)
The text was updated successfully, but these errors were encountered:
In Spark 3.2.1, a typo in the method signature for
DataSourceUtils.creteDateRebaseFuncInWrite
was fixed. apache/spark#34978 Of course, this change is neither backward nor forward compatible. Since the spark-avro jar is a compile-time dependency for ABRiS, other applications that create a fat-jar with the spark-avro v3.2.0 will experience aNoSuchMethodError
when running this against Spark 3.2.1 (see below), because the spark-avro jar is usually not part of the Spark distribution.The text was updated successfully, but these errors were encountered: