(Re-)Add content on generative AI

[tobyhodges]

[This repeats the changes made in #695 (then reverted in #696). I am sorry for the confusion! 😅 The pull request will stay open for at least a week, to give community members time to provide feedback and suggest improvements. Thanks for your patience @vahtras and other Maintainers ❤️ ]

This adds a new section to Built-in Functions and Help, titled "Other ways to get help" that discusses searching the internet, StackOverflow, talking to another person, and generative AI chatbots e.g. ChatGPT as possible ways to get more help when faced with errors while coding.

Some notes to guide feedback:

• I have tried to keep this as concise as possible, sticking to what I consider to be the most essential information only. But concede that it is still pretty wordy!
• These changes are guided by conversations within the community over recent months, including but not limited to the community discussion sessions summarised in a couple of recent blog posts (The Ethics of Teaching LLMs in Carpentries Workshops and Essential Knowledge and Misconceptions).

[github-actions]

Thank you!

Thank you for your pull request 😃

🤖 This automated message can help you check the rendered files in your submission for clarity. If you have any questions, please feel free to open an issue in {sandpaper}.

If you have files that automatically render output (e.g. R Markdown), then you should check for the following:

• 🎯 correct output
• 🖼️ correct figures
• ❓ new warnings
• ‼️ new errors

Rendered Changes

🔍 Inspect the changes: https://github.com/swcarpentry/python-novice-gapminder/compare/md-outputs..md-outputs-PR-697

The following changes were observed in the rendered markdown documents:

 04-built-in.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 md5sum.txt     |  2 +-
 2 files changed, 59 insertions(+), 1 deletion(-)

What does this mean?

If you have source files that require output and figures to be generated (e.g. R Markdown), then it is important to make sure the generated figures and output are reproducible.

This output provides a way for you to inspect the output in a diff-friendly manner so that it's easy to see the changes that occur due to new software versions or randomisation.

⏱️ Updated at 2025-03-28 15:02:55 +0000

[tobyhodges]

Quoting the comment received before #695 was merged. @rowleya wrote:

I personally think this is a very reasoned and reasonable approach and I think what you have written here relating both to AI and getting help certainly represents how I do things and also how I think about AI especially when learning. Good job!

[bkmgit]

A difficult topic. Licensing concerns are tricky.

It may be worth exploring how such tools are used ethically in educational settings, particularly for programming, and whether in future they could be incorporated into Carpentries lessons. One need not use LLMs, machine learning models and editor add ons can also be helpful. These might be explored in a separate lesson piloted through the incubator first though.

[tobyhodges]

Thanks @bkmgit. Indeed I would like to help get some lessons like you described through the Incubator. E.g. I think we could do a really nice follow-up to DC Image Processing, exploring ML methods for image analysis.

[brownsarahm]

do we want to link to sources on any of this? I can help provide them, but do not want to put them in if the goal is to not link out

more broadly, this might not fit here, but also, part of why learning to write code to do analyses is important is because LLMs do not reliably answer mathematical questions and for data privacy (as a counter to the idea a person might have to upload their data to a chatbot and ask it to do the analysis)

[dpshelio]

Very nice @tobyhodges! Thanks!

[tobyhodges]

@brownsarahm wrote

do we want to link to sources on any of this?

I considered this, but came down on the side of not including sources. If nothing else, just for the sake of keeping the focus on what we want to say in workshops, as opposed to which sources are the best fit for each point. But I could be easily persuaded in the opposite direction! If you or anyone else feels strongly, I can put some links in.

[mrawls]

I think you've threaded the needle reasonably well here. I am pretty strongly in the "don't use generative AI" camp, and I appreciate that you've incorporated context about pros, cons, and real world considerations alongside the blanket request to not use AI when you are learning to code because it defeats the purpose.

[drammock]

very nice work @tobyhodges. Appreciate all your efforts to include the community in this change.

[drammock]

this feels like more of a "sidebar" comment, since it's not about getting help when you're stuck. I wonder if it makes more sense to put it at the end of this section?

[tobyhodges]

I intended this to aid flow into the next paragraph. Roughly, something like:

• This is how the thing we are talking about now is similar to the things we talked about just before
• These are some of the ways in which people use this thing beyond what we already talked about
• But these are some ways in which that could be considered problematic

[drammock]

I see. In my view, the ethical problems apply regardless of whether you're using an LLM to get help when stuck, or using it to write whole programs from scratch (or refactor, translate code, etc). So I don't see why it needs to be right here in that regard.

[drammock]

If you take my suggestion of making the prior paragraph a "sidebar", then this should probably change to something like:

Suggested change

	However, there are drawbacks that you should be aware of.
	Additionally, there are drawbacks that you should be aware of.

[alee]

Thank you for this thoughtful and important addition @tobyhodges (and all the comments from the community have been really excellent as well). @swcarpentry/python-novice-gapminder-maintainers getting a lot of pings on this one 🤣

I'm running a clinic at the next CSDMS Annual Meeting on using LLMs for computational modeling so all of the points raised here are also helpful in thinking about how to refine and tailor that stream of work.

It would be great to include some summaries of the mailing list discussion here as well, or perhaps just a link to that entire thread?

[alee]

I think it may be important to emphasize that current LLMs do not appear to be "reasoning" by any sense of the word. They are statistical engines trained on enormous corpuses of data (internet+) that are very good at producing plausible, grammatically correct sentences that are statistically relevant to the given input (which you explain clearly later on). Would it be useful to be clear that current LLMs are nothing like AGI (which even if they did exist should also not be trusted unequivocally).

Though there is very interesting work being done by google deepmind in the area of neurosymbolic AI (alphageometry etc)...

I still find LLMs to be quite useful at search, summarization, explaining new concepts, rubber ducking, generating first drafts of code snippets, text, images, tests, build scaffolding and scripts, Dockerfiles, Apptainer recipes, k8s configuration, etc.

[alee]

I don't necessarily agree with the recommendation that learners not use LLMs during a workshop but this is a pretty soft position. Perhaps a brief interlude could have the instructor use a LLM live and demonstrate the back and forth assessment of the responses provided and techniques for effectively constraining and prompting an LLM. I have found tools like the the latest chatgpt, perplexity, notebooklm, github copilot, v0.dev, etc. to be valuable aids to my work but have also seen first-hand the effects of blind application of LLM responses without critical assessment; regardless people are using and will continue to use these tools so it may be important to demo some subset of good practices around them.

There was a "cooking" / "recipe" analogy raised on the mailing list that I don't think quite fits either, the main concern in my opinion is LLMs are statistical regurgitation-of-our-past-knowledge black boxes - https://garymarcus.substack.com/p/decoding-and-debunking-hard-forks does a good job of articulating these issues...

For a resources section I think this YouTube video (by Andrej Karpathy - worth following IMO) might be worth including, it is an excellent introduction to understanding what the current crop of LLMs actually do and how to more effectively use them:

https://www.youtube.com/watch?v=EWvNQjAaOHw

[tobyhodges]

Thanks to @alee for the suggestion to link to the ongoing discussion on the mailing list. For reasons that remain unclear to me, the thread is split into three (so far?) on the TopicBox site, but here they are:

1. Part 1
2. Part 2
3. Part 3

[mhagdorn]

@tobyhodges I also think this is a very considerate addition. I fully agree with the recommendations.

[brownsarahm]

Suggested change

	If you are comfortable, demonstrating how to work with an LLM, especially through the build in tools, could be a good way to end the workshop, in addition to pointing out other resources.

motivated by @alee's comment

[tobyhodges]

Can you elaborate on how this differs from the point already included on line 284 of the diff?

Demonstrate how you recommend that learners use generative AI.

My aim with this guidance to Instructros was to actively encourage different approaches based on the individual's level of expertise and comfort with using the technology

[tobyhodges]

Thanks everyone for your latest contributions. I really appreciate the feedback.

A heads-up that I plan to ask the Maintainers to merge this on Monday next week, so this is a last call for any "dealbreaking" reviews and suggestions!

[tobyhodges]

@swcarpentry/python-novice-gapminder-maintainers I think this is ready to merge now, if you are happy to do so. Thanks for your patience and support 🙌

[alee]

Thanks all for the stimulating and thoughtful discussion around this PR! 🥂