The Dawn of AI - Dr Simon McCallum

Today I am lucky enough to be down at the NZQA 'Assessment in the Age of AI' symposium. I will share my presentation later today. We started the day with a kick-ass keynote from Dr Simon McCallum. You can listen to him speak about the topic here:

How AI chatbots could shake up the world - item from TV show Q+A introducing AI and our keynote Simon McCallum

Below are my notes from his session. No apologies for typos and shorthand. ;) 

Understanding Generative AI

A large language model was initially designed with translation in mind. Translating requires understanding. "Meaning is Usage", mapping words and the connection between words. Some of what we mean is embedded in the words we use. LLM start by mapping words into vectors connecting relationships between words. Meaning in tokens, attention on windows, and building understanding. 



Source: https://www.analyticsvidhya.com/blog/2017/06/word-embeddings-count-word2veec/


Tokens are the basic units of text or code that an LLM AI uses to process and generate language. Tokens can be characters, words, subwords, or other segments of text or code, depending on the chosen tokenization method or scheme. Tokens are assigned numerical values or identifiers and are arranged in sequences or vectors, and fed into or outputted from the model. Tokens are the building blocks of language for the model.

Source: https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/tokens 


Everything is learned purely through communication and not lived experience. It's a shadow representation, but surprisingly correct. 


Prompt engineering 

Prompting is important. Triggering the right context. Too long and the model forgets. 

Pretext > <user query> > Posttest


Each output word is added to the input for the next calculation. 


Building context - Prompt Engineering

What they are good at

Translating level of language\Creating stories

Provide insight

Many tools using GPT backend - just add words to the context. 


Dump rubric into top of any writing. 


No longer just interacting with a large language model. 


LLM Plus code 

Systems built around the LLM

Pre-processing, guardrails, eval, constitutions

Post-processing, formatting, code eval., automation

Agent-based systems. 


Designing in the guardrails, and systems around AI that try to guide it. They are trying to build a conscience that the underlying system doesn't have. 


Additional processing 


GPT-4 with plugins


Add LLMs to any task 

Connect to the nest

Get information

Send information

Interact with people

AI Safety?


Can get actual references. 


AutoGPT/Agents

Uses python to call the LLM

Give it a task

It asked GPT to create a plan

The code then steps through executing the plan. 

Can be expensive $20 to solve a complex problem

But it can solve multistage challenges


Very scary as we don't know what plan it will have, it could make a mistake and do something weird. If I et it have full access it could destroy my stuff. What would this look like on a global scale? Rogue AI.


Cultural Simulation

Connect Sims to AI

They have motivations

They use language to communicate

Respond to language

You can talk to them.


Stable Diffusion - https://beta.dreamstudio.ai/generate

Dall-E 2 

Nvidia AI playground - https://www.nvidia.com/en-us/research/ai-playground/


Question about what is the creators but when combined with AI whose art is it?? The DNA of the original creator is still there. 


Not like humans


Blooms Taxonomy


Not ground up

AI can “understand” language

AI does not “understand” words

Failure is very different

Search and augmentation


It hasn’t had experiences it is remembering. A different intelligence to ours. 


Further reading: Learning in Artificial Intelligence: Does Bloom's Taxonomy Apply? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2891191


Uses for AI


Three main user groups

Students that use it to avoid learning

Students that are afraid of AI and avoiding
Students who use AI a lot and use it in interesting ways. 


Assessment

ChatGPT - NCEA L3

Bard & Bing - Level 7 - 9 in some areas

All work is now group work

Assess your contribution to the group

Motivation to learn


The path

We no longer know the path to productivity

What are foundational skills?

What must we do?

Is tradition a good indicator?


Augmenting

Augmenting humans

Measuring "authentic" human

What is assessment for? 


Credit farming

Transactional assessment

Alignment problems what the assessment tests and what the student can do


Types of assessment

Diagnostic - pre-learning

Formative - for learning

Summative - measurement/accreditation


Motivational - agentic, intrinsic, relevant, covert.


Authentic assessment 

Authentic to what?

Lost connection between task and time

Complex reasoning does not equal complex thought 

Task performance does not equal competence

Student + AI is hard to measure and changing. 


Replacing thinking

Concern that AI replaces thinking


Accelerating Learning


Good use cases 

Big picture - ask for detailsDetailed thinker - ask for big picture

Asking the 

Get tailored explanations


Educators becoming motivational speaker and inspiration - not content and planning. 


Policy changes

Treat AI as a co-author

Using AI as an editor

Justifying not using AI to help

Requiring AI evaluation for bias.


Treating AI as a horse

A great tool

Mostly riders responsibility

Acceptance that shit happens


AI for assessment

Flipped Exam - extracting information by 


2-5 years

Massive productivity shock

The collapse of the knowledge economy

Value human authenticity and connection

Emotional intelligence

Most academic learning becomes like exercise and dieting.


The Apathy epidemic

Obesity of the mind

Abdicating thinking to AIs.


Comments

Post a Comment

Popular posts from this blog

The National, Act and NZ First Coalition and what it means for education

Image
Photo Source Yesterday saw the announcement of a new government which came with a raft of new policies and ministerial appointments. The Minister of Education, as was expected, is National MP Erica Stanford. More interestingly was the number of policies and priorities two parties who only collectively gained 15.5% of the votes managed to get across the line.  The common threads across both Act and NZ First included some good old-fashioned back-to-basics, a focus on compulsory attendance and cracking down on truancy, increased focus on academic achievement and shifting the fees-free policy from the first of uni to the last. To be fair, none of these represent much of a departure from what National was touting anyway.  The interesting and, at times, worrying changes suggest Luxon was willing to let both Act and NZ First foist their ideologies on innocent schools and students for the sake of a "strong and stable" government.  Unsurprisingly Act got Partnership Schools across the
Read more

National, Act and the age of standardisation in education

Image
Chris and David discussing the joy of testing. Whilst we are still awaiting the outcome of special votes, it is pretty safe to assume that we are staring down the barrel of a National / Act Party government that will be bringing a very distinct flavour to the education landscape. Below is an outline of what each party promised for education.  Front and centre is a focus on "back to basics" and what looks like a whole lot more standardisation (from both National and Act) and testing to ensure we are all "teaching the basics brilliantly". Those of us in schools know that standardisation isn't necessarily the answer and testing didn't seem to turn the dial in the past either. That is the trouble when policies are designed to satisfy parent voters who just want their kids back in school and succeeding.  In their 100-day plan, National is promising to do the following in the education space: Require primary and intermediate schools to teach one hour a day each of
Read more

Assessment in the Age of AI - Claire Amos

Image
Below are the slides I shared at NZQA's 'Assessment in the Age of AI' symposium.   This is my brief overview of how we are approaching AI at Albany Senior High School and some of the concerns and some of thoughts I have about the topic.  In short, I am worried we are so damn busy we aren't giving it the attention it needs. I also worry that leaders will try and ignore and/or attempt to ban and that this ignores the fact that Pandora's Box is well and truly opened and this thing is developing faster than we can imagine and is increasingly woven into platforms and tools we already use. We are sitting at a fork in the path where both options lead to a world fundamentally altered by AI and we need to choose whether to work with it or stick our heads in the sand. That decision is going to have a tangible impact on how well our young people are going to be equipped to thrive in the future.  We need to embrace AI, and approach it with curiosity, criticality and care.  One
Read more