The Dawn of AI - Dr Simon McCallum

Today I am lucky enough to be down at the NZQA 'Assessment in the Age of AI' symposium. I will share my presentation later today. We started the day with a kick-ass keynote from Dr Simon McCallum. You can listen to him speak about the topic here:

How AI chatbots could shake up the world - item from TV show Q+A introducing AI and our keynote Simon McCallum

Below are my notes from his session. No apologies for typos and shorthand. ;) 

Understanding Generative AI

A large language model was initially designed with translation in mind. Translating requires understanding. "Meaning is Usage", mapping words and the connection between words. Some of what we mean is embedded in the words we use. LLM start by mapping words into vectors connecting relationships between words. Meaning in tokens, attention on windows, and building understanding. 



Source: https://www.analyticsvidhya.com/blog/2017/06/word-embeddings-count-word2veec/


Tokens are the basic units of text or code that an LLM AI uses to process and generate language. Tokens can be characters, words, subwords, or other segments of text or code, depending on the chosen tokenization method or scheme. Tokens are assigned numerical values or identifiers and are arranged in sequences or vectors, and fed into or outputted from the model. Tokens are the building blocks of language for the model.

Source: https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/tokens 


Everything is learned purely through communication and not lived experience. It's a shadow representation, but surprisingly correct. 


Prompt engineering 

Prompting is important. Triggering the right context. Too long and the model forgets. 

Pretext > <user query> > Posttest


Each output word is added to the input for the next calculation. 


Building context - Prompt Engineering

What they are good at

Translating level of language\Creating stories

Provide insight

Many tools using GPT backend - just add words to the context. 


Dump rubric into top of any writing. 


No longer just interacting with a large language model. 


LLM Plus code 

Systems built around the LLM

Pre-processing, guardrails, eval, constitutions

Post-processing, formatting, code eval., automation

Agent-based systems. 


Designing in the guardrails, and systems around AI that try to guide it. They are trying to build a conscience that the underlying system doesn't have. 


Additional processing 


GPT-4 with plugins


Add LLMs to any task 

Connect to the nest

Get information

Send information

Interact with people

AI Safety?


Can get actual references. 


AutoGPT/Agents

Uses python to call the LLM

Give it a task

It asked GPT to create a plan

The code then steps through executing the plan. 

Can be expensive $20 to solve a complex problem

But it can solve multistage challenges


Very scary as we don't know what plan it will have, it could make a mistake and do something weird. If I et it have full access it could destroy my stuff. What would this look like on a global scale? Rogue AI.


Cultural Simulation

Connect Sims to AI

They have motivations

They use language to communicate

Respond to language

You can talk to them.


Stable Diffusion - https://beta.dreamstudio.ai/generate

Dall-E 2 

Nvidia AI playground - https://www.nvidia.com/en-us/research/ai-playground/


Question about what is the creators but when combined with AI whose art is it?? The DNA of the original creator is still there. 


Not like humans


Blooms Taxonomy


Not ground up

AI can “understand” language

AI does not “understand” words

Failure is very different

Search and augmentation


It hasn’t had experiences it is remembering. A different intelligence to ours. 


Further reading: Learning in Artificial Intelligence: Does Bloom's Taxonomy Apply? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2891191


Uses for AI


Three main user groups

Students that use it to avoid learning

Students that are afraid of AI and avoiding
Students who use AI a lot and use it in interesting ways. 


Assessment

ChatGPT - NCEA L3

Bard & Bing - Level 7 - 9 in some areas

All work is now group work

Assess your contribution to the group

Motivation to learn


The path

We no longer know the path to productivity

What are foundational skills?

What must we do?

Is tradition a good indicator?


Augmenting

Augmenting humans

Measuring "authentic" human

What is assessment for? 


Credit farming

Transactional assessment

Alignment problems what the assessment tests and what the student can do


Types of assessment

Diagnostic - pre-learning

Formative - for learning

Summative - measurement/accreditation


Motivational - agentic, intrinsic, relevant, covert.


Authentic assessment 

Authentic to what?

Lost connection between task and time

Complex reasoning does not equal complex thought 

Task performance does not equal competence

Student + AI is hard to measure and changing. 


Replacing thinking

Concern that AI replaces thinking


Accelerating Learning


Good use cases 

Big picture - ask for detailsDetailed thinker - ask for big picture

Asking the 

Get tailored explanations


Educators becoming motivational speaker and inspiration - not content and planning. 


Policy changes

Treat AI as a co-author

Using AI as an editor

Justifying not using AI to help

Requiring AI evaluation for bias.


Treating AI as a horse

A great tool

Mostly riders responsibility

Acceptance that shit happens


AI for assessment

Flipped Exam - extracting information by 


2-5 years

Massive productivity shock

The collapse of the knowledge economy

Value human authenticity and connection

Emotional intelligence

Most academic learning becomes like exercise and dieting.


The Apathy epidemic

Obesity of the mind

Abdicating thinking to AIs.


Comments

Post a Comment

Popular posts from this blog

Education in Aotearoa - the fairy tale we are being fed and the fairy tale we need.

Image
Maybe it's the English teacher in me, but I can't help coming back to two fairy tales when I think about what is going on in education in Aotearoa at the moment. One fairy tale we seem to be currently living through, 'Chicken Little' and one fairy tale we could very dearly learn from 'Goldilocks and the Three Bears'.  Part One: Chicken Little and Maths education in NZ - There's no crisis like a manufactured crisis!   The fairy tale of Chicken Little, also known as "Henny Penny" or "The Sky is Falling," tells the story of a small, fearful chicken who is struck on the head by a falling acorn. Mistaking the acorn for a piece of the sky, Chicken Little believes that the sky is falling and sets out to warn the king. Along the way, Chicken Little meets other animals like Henny Penny, Ducky Lucky, and Goosey Loosey, who join the journey to alert the king. However, they are eventually deceived by a clever fox, Foxy Loxy, who invites them into his...
Read more

The Principal Diaries - The Annual Plan and putting the plan into action!

Image
It was an interesting process pulling together my first annual plan as Principal at Albany Senior High School. School Charters and Annual Plans are interesting beasts. To be honest, they appear to be box ticking waffle and weasel words which gather cyber dust in some virtual filing system. My belief was that a plan can be bloody useful, but only if it was actually designed to be useful. For that reason, I set about researching and reading as many as I could lay my hands on. What I found, for the most part, was (I thought) unnecessarily long winded and either attempted to capture so much in so much detail that they seemed insurmountable or so vague they read like paraphrased business as usual. In the end, the easiest approach was to go to The University of Auckland Centre for Educational Leadership to look at their resources and templates and have a crack of building something from scratch. The following is my attempt to craft an Annual Plan (this obviously doesn't include the br...
Read more

EDUCATION - Where there are things that other countries have done well, we should steal it!

Image
Source: NEWSHUB.CO.NZ In an interview with NewsHub on Monday the Prime Minister Christopher Luxon said his Coalition Government has set some ambitious goals they want New Zealanders to focus on and will take inspiration from other countries to achieve them. "We are here to improve the country by these targets and as a result, we look at whatever is working around the world," Luxon said. He talked about three countries in particular due to them being a similar size and examples of success. Going so far as to say “Where there are things that other countries have done well, we should steal it!” Concerns about " used futures " put to one side, let's investigate this idea a little further. So who were the three counties he named? Estonia, Ireland and Singapore. So that got me thinking, what does education look like in these countries? And when and how do we start our plundering?  Singapore: Holistic Education: Emphasising the development of students' character, ...
Read more