first-test-with-supervisor-bot-in-meeting

I prefaced the bot and its reasonable simplicity and faults at this stage with my supervisors before we started the demo.

However it became really clear how bad it is, it’s contributions don’t really challenge anything and feel seemingly unconnected to anything that we have actually really say and more just saying stuff that is knows from the context.

a good question I got was: is the supervisor bot a bit like an echo chamber as it is trained on my data?

this is something to really bear in mind, and I need to do a better job of engineering the actual LLM architecture I think to better engage with inputs. Something we also discussed was that perhaps the real ‘unique sellin point’ of a human supervisor is that they have lived experience and can actually contribute advice from this grounded place.

We joked maybe the LLM needs to have the experience of doing a PhD before its useful, and as I do like to follow absolutely wacky thought experiments, I am intrigued by what that might/could mean in practical real terms. There is the LLM research project by Karpathy that could contribute to fleshing out this a bit more…

• read the LLM paper by Karpathy as starting point, how might this be (can it be extrapolated to a less scientific approach). #milestone

I need to add this to the reflections-on-supervisor-bot-v1point5

other potentially related papers:
The AI Scientist (Lu et al., 2024)
Disembodied creativity in generative AI: prima facie challenges and limitations of prompting in creative practice

foundational texts:
Dreyfus, “What Computers Still Can’t Do” (1972, revised 1992)

Schön, “The Reflective Practitioner” (1983)?