add documentation for Abris 5 #253
Conversation
cerveada
commented
Nov 23, 2021
cerveada
commented
- remove unused dependencies
- remove prints in test
- remove unused dependencies - remove prints in test
README.md
Outdated
| This means the Spark schema doesn't need to be changed (to non-nullable columns) for avro conversion to succeed. | ||
|
|
||
| ## Confluent Schema Registry Version | ||
| Abris by default uses Confluent client version 5.3.4, but any newer version should work. |
There was a problem hiding this comment.
Can any newer version of Confluent work? I thought the compatibility problem between Confluent v5.4.+ and Spark will only be resolved with Avro 1.10 in Spark 3.2.0
There was a problem hiding this comment.
From the Abris side it should work, but yes there might be issues elsewhere. I think that it may work if you changed the avro in Spark to the one compatible with schema registry. I am not sure how hard is it to change the libraries that spark is using.
Maybe I should remove that part and just keep the confluent 5.3 as recommended version until Spark 3.2-Abris is ready.
There was a problem hiding this comment.
I think the situation has not changed compared to #165. If you look at the issue there, we figured out that the problem was the Avro version. Since we didn't follow up on a solution to that, I wouldn't add it here.
There was a problem hiding this comment.
Ok, then I wall remove it from release notes as well, so people don't think that everything will work
* add documentation for Abris 5 - remove unused dependencies - remove prints in test
