Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduction of results for ESOL and FreeSolv #9

Open
subhalingamd opened this issue Jul 10, 2023 · 1 comment
Open

Reproduction of results for ESOL and FreeSolv #9

subhalingamd opened this issue Jul 10, 2023 · 1 comment

Comments

@subhalingamd
Copy link

Hi, thanks for releasing the pre-trained model and the code. Could you share the scripts used for fine-tuning on ESOL and FreeSolv data?

I am more interested in the hyper-parameters. I made the scripts similar to the Lipophilicity script but got way higher RMSE (e.g., more than 1 in case of FreeSolv).

Thanks.

@GintasKam
Copy link

could the RMSE's in the paper have been computed on the standardized values rather than the original ones?.. I think that was also the issue in another (BARTSmiles) llm paper that showed order-of-magnitude improvements in regression tasks.

for example, in the MolFormer repositories' data the lipophilicity values seem to be standardized (centered around 0 and all with ~10 decimal points) whereas the MoleculeNet datasets are in the 0-7 range and fewer decimal points. clarification around the regression datasets' treatment would be very appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants