GPT-4 – How does it work, and how do I build apps with it?

oh that's interestingall right well this is a cs50 tech talkthank you all so much for coming soabout a week ago we circulated theGoogle form as you might have seen at 1052 a.m and by like 11 52 a.m We had 100RSVPs which I think is sort of Testamentto just how much interest there is inthis world of AI and open Ai and GPT atGPT and the like and in fact if you'resort of generally familiar with whateveryone's talking about but you haven'ttried it yourself like this is the URLwhich you can try out this tool thatyou've probably heard about chat GPT youcan sign up for a free account there andstart tinkering with what everyone elsehas been tickering with and then ifyou're more of the app minded type whichyou probably are if you are here with ustoday open AI in particular has its ownlow-level apis via which you canintegrate AI into your own software butof course as is the case in computerscience there's all the moreabstractions and services that have beenbuilt on top of these Technologies andwe're so happy today uh to be joined byour friends from McGill University andsteamship uh sill and Ted from whomyou'll hear in just a moment to speak tous about how they are making it easierto build to deploy to share applicationsusing some of these very sameTechnologies so our thanks to them forhosting today our friends at PlimptonJenny Lee and alumna who's here with ustoday but without further Ado allow meto turn things over to Ted and sill andpizza will be served shortly after 1 pmoutsideall right over to you Ted thanks a lothey everybody it's great to be here Ithink we've got a really good talk foryou today still is going to provide someresearch grounding into how it all workswhat's going inside the brain of GPT aswell as other language models and thenI'll show you some examples that we'reseeing on the ground of how people arebuilding apps and what apps tend to workin the real world so our perspective iswe're building AWS for AI apps so we getto talk to a lot of the makers who arebuilding and deploying their apps andthrough that see both the experimentalend of the spectrum and also see whatkinds of apps are getting pushed outthere and turned into companies turnedinto side projects we did a coolhackathon yesterday many thanks to toNeiman to David Malin and cs50 forhelping us put all of this together toHarvard for hosting it and there weretwo sessions lots of folks built thingsif you go to steamship.com hackathonyou'll find a lot of guides a lot ofprojects that people built and you canfollow along we have a text guide aswell just as a quick plug for that ifyou want to do it remotely or on yourownum so to tee up still we're going totalk about basically two things todaythat I hope you'll walk away with andreally know how to then use as youdevelop and as you Tinker one is what isGPT and how is it working get a goodsense of what's going on inside of itother than as just this magical machinethat predicts things and then two is howare people building with it and thenimportantly how can I build with it tooif you're a developer and if you havecs50 background you should be able topick things up and start building somegreat apps I've already met some of thecs50 grads yesterday and the things thatthey were doing were pretty amazing sohope this is useful I'm going to kick itover to sill and talk about some of thetheoretical front of GPTyeah so thank you Ted um my name is soI'm a graduate student in the digitalHumanities at McGill I study literatureand computer science and Linguistics inthe same breath and I've published someresearch over the last couple of yearsexploring what is possible with languagemodels and culture in particular and myhalf or whatever of the presentation isto describe to you what is GPT that'sreally difficult to explain in 15minutes and there are even a lot ofthings that we don't know but a good wayto approach that is to First considerall the things that people call GPT byor descriptors so you can call themlarge language models you can call themUniversal approximators from computerscience you can say that that it is agenerative AIwe know that they are neural networks weknow that it is an artificialintelligence to some it's a simulator ofculture to others it just predicts textit's also a writing assistant if you'veever used Chachi PT you can plug in apit of your essay get some feedback it'samazing for that it's a Contentgenerator people use it to docopywriting jasper.ai pseudorite Etcit's an agent so the really hot thingright now if you might have seen it onTwitter Auto GPT baby AGI people aregiving these things tools and lettingthem run a little bit free in the wildto interact with the world computers etcwe use them as chat Bots obviously andthe actual architecture is a Transformerso there's lots of ways to describe GPTand any other one of them is a reallyperfectly adequate way to begin theconversation but for our purposes we canthink of it as a large language modeland more specifically a language modeland a language modelis a model of language to if you allowme the tautology but really what it doesis it produces a probabilitydistribution over some vocabulary so letus imagine that we had the task ofpredicting the next word of the sequenceI am so if I give a neural networkthe words I am what of all words inEnglish is the next most likely word tofollow that at its very core is what GPTis trained to answerand how it does it is it has avocabulary of 50 000 words and it knowsroughly given the entire internet whichwords are likely to follow other wordsof those 50 000 in some sequence up totwo thousand words up to four thousandup to eight thousand and now up tothirty two thousand three pt4 so yougive it a sequence here I am and overthe vocabulary of 50 000 words it givesyou the likelihood of every single wordthat follows so here it's I am perhapsthe word happiest fairly frequent sowe'll get that high probability if welook at all words all utterances ofEnglish it might be I am sad maybethat's a little bit less probable I amschool that really should be at the endbecause I don't think anybody would eversay that I am Bjork that's a little bitit's not very probable but it's lessprobable than happy sad but there'sstill some probability attached to itand when we say it's probable that'sliterally a percentage that's likehappy follows I am maybe like fivepercent of the time sad photos I ammaybe two percent of the time orwhatever so for every wordthat we give GPTit tries to predict what the next wordis across 50 000 words and it givesevery single one of those 50 000 wordsuh number that reflects how probable itisand the Really magical thing thathappens is you can generate new text soif you give GPT I am and it predictshappy as being the most probable wordover fifty thousand you can then appendit to I am so now you say I am happy andyou feed it into the model again yousample another word you feed it into themodel again and again and again andagain and there's lots of different waysthat I am happy I am sad can go and youadd a little bit of Randomness and allof a sudden you have a language modelthat can write essays that can talk anda whole lot of things which is reallyunexpected and something that we didn'tpredict even five years ago so this isall relevant and if we move onas we scale up the model and we give itmore compute in 2012 Alex that came outand we figured out we can give the modeluh we can run the model on gpus so wecan speed up the process we can give themodel lots of information downloadedfrom the internet and it learns more andmore and more and the frequent theprobabilities that it gives you getbetter as it sees more examples ofEnglish on the internet so we have totrain the model to be really largereally wide and we have to train it fora really long time and as we do that themodel gets more and more better andexpressive and capable and it also getsa little bit intelligent and for reasonswe don't understandso but the also the issue is thatbecause it learns to replicate theinternet it knows how to speak in a lotof different genres of text and a lot ofdifferent registers if you begin theconversation like chat GPT can youexplain the moon landing to a six yearold in a few sentences gpt3 this is anexample drawn from the instruction PTApaper from openaigpt3 would have just been like okay soyou're giving me an example like explainthe moon landing to a six-year-old I'mgoing to give you a whole bunch ofsimilar things because those seem verylikely to come in a sequence it doesn'tnecessarily understand that it's beingasked a question has to respond with ananswer gpt3 did not have that apparatusthat interface for responding thequestions andthe scientists at openai came up withthe solution and that's let's give it awhole bunch of examples of question andanswers such that we first traded on theinternet and then we train it with awhole bunch of questions and answerssuch that it has the knowledge of theinternet but really knows that it has tobe answering questions and that is whenchat GPT was born and that's when itgained 100 million users in one month Ithink it'd be tick tock's record at 20million in one month it was a huge thingand for a lot of people they went ohthis thing is intelligent I can answer Ican ask it questions it answers back wecan work together to come to a solutionand that's because it's still predictingwords it's still a language model but itknows the predicts wordsin the framework of a question andanswer so that's what a prompt is that'swhat instruction tuning is that's a keywordthat's whatrlhf is if you've ever seen that acronymreinforcement alignment with humanfeedback and all those combined meansthat the models that are coming outtoday the types of language predictorsthat are coming out today work tooperate in a q a formgpt4 exclusively only has the Alignmodel available and this is a reallygreat solid foundation to build onbecause you can do all sorts of thingsyou can ask Chachi PT can you do thisfor me can you do that for me you mighthave seen that open AI has allowedplug-in access to chat CPT so it canaccess Wolfram it can search the web itcan search it can do instacart for youit can look up recipesonce the model knows that not only ithas to predict language but that it hasto solve a problemand the problem here being give me agood answer to my question it's suddenlyable to interface with the world in areally solid way and from there onthere's been all sorts of tools thatbuild on this q a form that chatgpt usesyou have Auto GPT you have Lang chainyou haveuh react there's a react paper where alot of these come from andturning the model into an agent withwhich to achieveany ambiguous goal is where the futureis going and this is all thanks toinstruction tuning and with that I thinkI will hand it off to Ted who will begiving a demo or something along thoselines for how to use GPT as a agentsoall right so I'm a super replied guy Ikind of look at things and think okayhow can I like to add this Lego add thatLego and clip them together and buildsomething with it and right now you knowif you look back in computer sciencehistory when you look at the kinds ofthings that were being done in 1970right after Computing was invented themicroprocessors were invented peoplewere doing research like how do I sort alist of numbers and that was meaningfulwork and importantly it was work that'saccessible to everybody because nobodyknows what we can build with this newkind of oil this new kind of electricitythis new kind of unit of computationwe've created and anything was game andanybody could participate in that gameto figure it out and I think one of thereally exciting things about GPT rightnow is yes in and of itself it's amazingbut then what could we do with it if wecall it over and over again if we buildit into our algorithms and start tobuild it into broader software so theworld really is yours to figure outthose fundamental questions about whatcould you do if you could scriptcomputation itself over and over againin the way that computers can do notjust talk with it but build things atopit so we're a hosting company we hostapps and these are just some of thethings that we see I'm going to show youdemos of this with code and try toexplain some of the thought process butI wanted to give you a high level ofoverview of you've probably seen theseon Twitter but kind of when it all sortsout to the top these are some of thethings that we're seeing built anddeployed with language models todaycompanionship that's everything from Ineed a friend do I need a friend with apurpose I want a coach I want somebodyto tell me go to the gym and do theseexercises I want somebody to help mestudy a foreign language questionanswering this is a big one this iseverything from your Newsroom having aslack bot that helps assist you doesthis article conform to the StyleGuidelines of our Newsroom all the waythrough to and you need help on myhomework or hey I have some questionsthat I want you to ask Wikipedia combineit with something else synthesize theanswer and give it to me utilityfunctions I would describe this as asthere's a large set of things for whichhuman beings can do them if only orcomputers could do them if only they hadaccess to language computation languageknowledge an example of this would beread every tweet on Twitter tell me theones I should read that way I only getto read the ones that actually makesense to me and I don't have to skimthrough the rest creativity imagegeneration text generation storytellingproposing other ways to do things andthen these wild experiments and kind ofbaby AGI as people are calling them inwhich the AI itself decides what to doand is self-directed so I'll show youexamples of many of these and what thecode looks like and if I were you Iwould think about these as categorieswithin which to both think about whatyou might build and then also seek outstarter projects for how you might goabout building them onlineall right so I'm just going to divestraight into demos and code for some ofthese because I know that's what'sinteresting to see as fellow Builderswith a high level diagram for some ofthese as to how it works soapproximately you can think of acompanionship bot as a friend that has apurpose to you and there are many waysto build all of these things but one ofthe ways you can build this is simply towrap GPT or a language model in anendpoint that additionally injects intothe prompt some particular perspectiveor some particular goal that you want touse it really is that easy in a way butit's also very hard because you need toiterate and engineer The Prompt so thatit consistently performs the way youwant it to perform so a good example ofthis is something somebody built intohackathon yesterday and I just wanted toshow you uh the project that they builtit was a mandarin idiom coach and I'llshow you what the code looked like firstI'll show you the Demo FirstI think I already pulled it uphere we go so the the buddy that thisperson wanted to create was a friendthat if you gave it a particular problemyou were having it would pick a Chineseidiom a four character Chung you thatdescribe poetically like here's a aparticular way you could say this and itwould tell it to her so that the personwho built this was studying Chinese andshe wanted to learn more about it um soI might say something like I'm feelingvery sadand it would think a little bitand if everything's up and running itwill generate one of these fourcharacter phrases and it will respond toit uh with an example now I don't knowif this is correct or not so if somebodycan call me out if this is actuallyincorrect you please please call me outum and it will then finish up withsomething encouraging saying hey you cando it I know this is hard keep going solet me show you how they built this andI uhpulled up the coderight here so this was the particularstarter replit that folks were using inthe hackathon yesterday and we we pulledthings up into basically you have awrapper around GPTand there's many things you could do butwe're going to make it easy for you todo two things one of them is to injectsome personality into the promptand I'll explain what that prompt is ina second and then the second is ADDtools that might go out and do aparticular thing search the web orgenerate an image or add something to adatabase or fetch something from adatabase so having done that now youhave something more than GPT now youhave GPT which we all know what it isand how we can interact with it butyou've also added a particular lensthrough which it's talking to you andpotentially some tools so thisparticular Chinese tutor all it took tobuild that was four lines so here's aquestion that I think is is frying theminds of everybody in the industry rightnowso is this something that we'll all docasually and nobody really knows well wejust all say in the future to the llmhey for the next five minutes pleasetalk like a teacher and maybebut also definitely in the meantime andmaybe in the future it makes sense towrap up these personalized endpoints sothat when I'm talking to GPT I'm notjust talking to GPT I have a whole Armyof different buddies of differentcompanions that I can talk to they'rekind of human and kind of talk to meinteractively but because I pre-loadedthem with hey by the way you particularI want you to be a kind helpful Chineseteacher that responds to every situationby explaining the changu that fits itspeak in English and explain the Chungin its meaning then provide a note ofencouragement about learning languageand so just adding something like thateven if you're a non-programmer you canjust type deployand it'll pop it up to the webit'll take it over to a telegram botthat then you can even interact with heyI'm feeling too busyand interact with it over telegram overthe web and this is the kind of thingthat's now Within Reach for everybodyfrom a CS 101 grad sorry I'm using thegeneral purpose framing all the waythrough to Professionals in the industrythat you can do just with a little bitof manipulation on top of sort of thisraw unit of conversation andintelligenceso companionshipis is one of the first Common uh typesof apps that we're seeingso a secondkind of app that we're seeing and thisblew up if for those of you who are onuh kind of Twitter followers this blewup I think the last few months isquestion answering and I want to unpacka couple of different ways this can workbecause I know many of you have probablyalready tried to build some of thesekinds of apps there's a couple ofdifferent ways that it works the generalframework isa user queries GPT and maybe it hasgeneral purpose knowledge maybe itdoesn't have general purpose knowledgebut what you want it to say back to youis something specific about an articleyou wrote or something specific aboutyour course syllabus or somethingspecific about a particular set ofdocuments from the United Nations on aparticular topic and so what you'rereally seeking is what we all hoped thecustomer service spot would be likewe've all interacted with these customerservice Bots and we're kind of Smashingour heads on the keyboard as we do itbutpretty soon we're going to start to seevery high fidelity Bots that interactwith us comfortably and this isapproximately how to do it as anengineer so here's your game plan as anengineer step onetake the documents that you want it torespond tostep two cut them up now if you're anengineer this is going to Madden you youdon't cut them up in a way that youwould hope for example you could cutthem up into clean sentences or cleanparagraphs or or semantically coherentsections and that would be really nicehonestly the way that most folks do itand this is a simplification that tendsup tends to be just fine is you windowyou have a sliding window that goes overthe document and you just pull outfragments of texthaving pulled out those fragments oftext you turn them into something calledan embedding Vector so an embeddingVector is a list of numbers thatapproximate some point of meaningso you've already all dealt withembedding vectors yourself in regularlife and the reason you have and I knowyou have is because everybody's orderedfood from Yelp before so when you orderfood from Yelp you look at what genre ofrestaurant is it is it in a pizzarestaurant is it an Italian restaurantis it a Korean barbecue place you lookat how many stars does it have one twothree four five you look at where is itso all of these you can think of aspoints in space dimensions in spaceKorean barbecue restaurant four starsnear my house it's a threethree number vectorthat's all this is so this is a thousandnumber vector or a ten thousand numberVector different models producedifferent size vectors all it is ischunking pieces of text turning it intoa vector that approximates meaning andthen you put it in something called avector database and a vector database isjust a database that stores numbersbut having that database now when I aska questionI can search the database and I can sayhey the question was what does cs50teachwhat pieces of text in the database havevectors similar to the questionwhat does cs50 teach and there's allsorts of tricks and Empires being madeon refinements of this General approachbut at the end you the developermodel it simply as thusand then when you have your query youembed it you find the document fragmentsand then you put them into a prompt andnow we're just back to the personalitythe thecompanionship Bots now it's just aprompt and the prompt is you're anexpert in answering questions pleaseansweruser provided question using Sourcedocuments results from the databasethat's itso after all of these Decades ofengineering these customer service spotsit turns out with a couple of lines ofcode you can build this so let me showyou I made one just before the classwith the cs50 syllabus so we canpull that upand I can say I I added the PDF righthere so I just I searched I don't knowif I apologize I don't know if it's anaccurate or recent syllabus I justsearched the web for cs50 syllabus PDF Iput the URL in here uh it it loaded itinto here this is just a like a hundredline piece of code deployed that willnow let me talk to it and I can say whatwill cs50 teach meso under the hood now what's happeningis exactly what that slide just showedyou it takes that question what willcs50 teach me it turns it into a vectorthat Vector approximates without exactlyrepresenting the meaning of thatquestion it looks into a vector databasethat steamship hosts of fragments fromthat PDFand then it pulls out a document andthen passes it to a prompt that says heyyou're an expert at answering questionssomeone has asked you what the cs50teach please answer it using only thesource documents and Source materialsI've provided now those Source materialsmaterials are dynamically loaded intothe prompt it's just basic promptengineering and I want to keep harpingback onto that what's amazing aboutright now is Builders is that so manythings just boil down to very creativeTacticalrearrangement of prompts and then usingthose over and over again in analgorithm and putting that into softwareso the result and again it could belying it could be making things up itcould be hallucinating is cs50 willteach students how to thinkalgorithmically and solve problemsefficiently focusing on topics such asabstraction and then it Returns thesource document from which it was foundso this is another big bigory of whichthere are tonsof potential applications because youcan repeat for each context you know youcan create arbitrarily many of theseonce it's software because once it'ssoftware you can just repeat it over andover again so for your dorm for yourclub for your slack for your telegramyou can start to begin putting pieces ofinformation in and then responding to itand it doesn't have to be documents youcan also load it straight into thepromptI think I have it pulled up here and ifI don't I'll just skip itoh here we goone other way you can do questionansweringbecause I I think it's healthy to alwaysencourage the simplest possible approachto somethingyou don't need to engineer this giantsystem it's great to have a databaseit's great to use embeddings it's greatto use this big approach it's fancy itscales you can do a lot of things butyou can also get away with a lot by justpushing it all into a prompt and as a asan engineer I'm you know yes one of ourteammates here always says likeEngineers should aspire to be lazy and Icouldn't agree more you as an engineershould want to set yourself up so thatyou can pursue the lazy path tosomethingso here's how you might do theequivalent of a question answeringsystem with a prompt alone let's say youhave 30 friends and each friend is goodat a particular thing or you can youknow this isomorphic to many otherproblems you can simply just say hey Iknow certain things here's the things Iknowa user is going to ask me somethinghow should we respondand then you load that into an agentthat agent has access to GPTyou can shift deploy it and now you'vegot a bot that you can connect totelegram you can connect to slack andthat bot now it won't always give youthe right answer because at a certainlevel we can't control the variance ofthe model underneathbut it will tend to answer with respectto this list and and the degree to whichit tends to is to a certain extentsomething that both industry is workingon to just give everybody as a capacitybut also you doing prompt engineering totighten up the the error bars on itso I'll show you just a few moreexamples uh and then in about eightminutes I'll turn it over to questionsbecause I'm sure you've got a lot abouthow to build things so just to give youa sense of of where we areforeignthis is one I don't have a demo for youbut if you were to come to me and youwere to say TedI want a weekend Hustle Man what shouldI build holy moly there are a set ofapplications that I would describe asutility functions I don't like that namebecause it doesn't sound exciting andthis is really excitingand it's it's low hanging fruits thatautomate tasks that require basiclanguage understanding so examples forthis are generate a unit tests I don'tknow how many of you have ever beenwriting tests and you're just like ohcome on I can get through this I can getthrough this if you're a person wholikes writing tests you're a luckyindividuallooking up the documentation for afunction rewriting a function makingsomething conform to your companyguidelines doing a brand checkall of these things are things that arekind ofrelatively context-free operations orscoped context operations on a piece ofinformation that requires linguisticunderstandingand really you can think of them assomething that is now available to youas a software Builder as a weekendproject Builder as a startup Builderand you just have to build the interfacearound it andpresented to other people in a contextin which it's meaningful them for themto consume and so the space of this isextraordinary I mean it's the space ofall human endeavor now with this newtool I think is the way to the way tothink about it people often joke abouthow in when you're building a companywhen you're building a project you don'twant to start with a hammer because youwant you want to start with a probleminstead and it's generally true but myGod like we've just got a really coolnew hammer and to a certain extent Iwould encourage you to at least casuallyon the weekends run around and hit stuffwith it and see what can happen from aBuilders from a tinkerers fromexperimentalists point of viewand then the finalcreativity this is another huge mega appnow I'm primarily live in the text worldand so I'm going to talk abouttext-based things I think so far this ismostly uh been growing in the imageryworld because we're such visualcreatures and the the images you cangenerate are just staggering with AIcertainly brings up a lot of questionstoo around IP and artistic stylebut the template for this if you're abuilder that we're seeing in in the wildis approximately the following and thething I want to point out is domainknowledge here this is really thepurpose of this slide is to to touch onthe importance of the domain knowledgeso many peopleapproximately find the creative processas follows come up with a big ideaover generate possibilitiesedit down what you over generated repeatright like anybody who's been a writerknows when you write you write way toomuch and then you have to delete lots ofit and then you revise and you write waytoo much and you have to delete lots ofit this particular task is fantastic forAI one of the reasons it's fantastic forAI is because it allows the AI to bewrong you know you've pre-agreed you'regoing to delete lots of it and so if youpre-agree hey I'm just going to buildyou know generate five possibilities ofthe story I might tell fivepossibilities of the advertisingheadline five possibilities of what Imight write what I might write my thesison you pre-agreed it's okay if it's alittle long because you are going to bethe editor that steps in and and here'sthe thing that you really should bringto the table is don't think about thisas a technical activity think about thisas your opportunity not to put GPT inchargeinstead for you to grasp the steeringwheel tighter I think at least in pythonor the language you're using to programbecause you have the domain knowledge towield GPT in the generation of those solet me show you an example of what Imean by that sothis is a a cool app that someonecreated for the writing Atlas project sowriting Atlas is a set of short storiesand you can think of it as Goodreads forshort stories so you can go in here youcan browse different stories and thiswas something somebody created where youcan type in a story a description thatyou like and this is going to take abouta minute to generate so I'm going totalk while it's generating andwhilewhile it's working what it's doing andI'll show you the code in a second isit's searching through the collection ofstories for similar stories and here'swhat the domain knowledge part comes inthen it uses GPT to look at what it wasthat you wanted and use knowledge of howan editor how a Bookseller thinks togenerate a set of suggestionsspecifically through the lens of thatperspective with the goal of writingthat beautiful handwritten note that wesometimes see in a local bookstoretacked on underneath a book and so itdoesn't just say hey you might like thishere's a general purpose reason why youmight like this but specifically here'swhy you might like this with respect towhat you gave it it's either stallingout or it's taking a long time oh therewe goso here's its suggestionsand in particular these things these arethings that only a human could know atleast for now uh two humans specificallythe human who said they wanted to read astory that's the text that came in andthen the human who added domainknowledge to script a sequence ofinteractions with the language model sothat you could provide very targetedreasoning over something that wasinformed by that domain knowledge so forthese utility apps bring your bring yourdomain knowledgelet me actually show you how this looksin code because I think it's it's usefulto see how simple and accessible this isthis is reallya set of prompts so why might theylike a particular location well here'sthe prompt that did that this is an opensource projectand and it has a bunch of examples andthen it says well here's the one thatwe're interested inhere's the audience here's a couple ofexamples of why might people like aparticular thing in terms of audienceit's just another promptsame for topic same for explanation andif you go down here and look at how itwas donesuggesting the story is what is thisline 174 to line 203 it really is andagain like over and over again I want toimpress upon you this really is WithinReach it's really justwhat 20 odd lines of Step One search inthe database for similar stories steptwo given that I have similar storiespull out the data step three with mydomain knowledge in Python now run theseprompts step four prepare that into anoutput so the thing we're scriptingitselfis some approximation of human cognitionif you're willing to go theremetaphorically we're not you know we'renot sure I'm not going to weigh in on onwhere we are in the is open AI uha life formargumentall right uh one kind of really far outthere thing and then I'll uh tie it upfor questions because I know there'sprobably a lot and I also want to makesure you get a great pizza in yourbellies and that is a baby AGI Auto GPTis what you might have heard them calledon Twitter I think of them as multi-stepplanning bots so everything I showed youso far was approximately One-Shotinteractions with GPTso this is the user says they wantsomethingand then either python mediatesinteractions with GPT or GPT itself doessome things with the inflection of apersonality that you've added from someprompt engineering really usefulpretty easy to control if you want to goto production if you want to build aweekend project if you want to build acompany that's a great way to do itright nowthis is wild and if you haven't seenthis stuff on Twitter I would definitelyrecommend going to search for it this iswhat happens the simple way to put it isif you put GPT in a for Loop if you letGPT talk to itself and then tell itselfwhat to dosoit it's an emergent Behavior like andlike all emergent behaviors it startswith a few simple steps the Conway'sGame of Life manyelements of reality turn out to be mathequations that fit on a t-shirt but thenwhen you play them forward in time theygenerate DNA they generate human life sothis is approximatelystep one take a human objectivestep two your first task is to writeyourself a list of steps and here's thecritical part repeatnow do the list of steps now you have toembody your agent with the ability to dothings so it's really only limited to dowhat you give it the tools to do andwhat it has the skills to do soobviously this is still very much a setof experiments that are running rightnow and and but it's something thatwe'll see unfold over the coming yearsand this is the scenario in which pythonstops becoming so important becausewe've given it the ability to actuallyself-direct what it's doing and then itfinally gives you a result and I want togive you an example still of just againimpressing upon you how much of this isprompt engineering which is wild howlittle code this is let me show you whatbaby AGI looks likeso here is a baby AGI that you canconnect to Telegramand this is an agent that has two toolsso I haven't explained to you what anagent is I haven't explained to you whattools are I'll give you a quick onesentence description an agent is just aword to mean GPT plus some bigger bodyin which it's living maybe that body hasa personality maybe it has tools maybeit has python mediating its experiencewith other things tools are simply waysin which the agent can choose to dothings like imagine if GPT could sayorder a pizza and instead of you seeingthe text order a pizza that caused thepizza to be ordered that's a toolso these are two tools it has one toolis generated to-do list one tool is do asearch on the weband then down hereit has a a prompt saying hey your goalis to build a task list and then do thattask list and then this is just placedinto a harness that does it over andover again so after the next task kindof uncue the results of that task andand keep it goingand so in doing that you get thiskickstarted Loop where essentially youkick start it and then the agent istalking it to itself talking to itselfso this unless I'm wrong I don't thinkthis has yet reached production in termsof what we're seeing in the field of howpeople are deploying software but if youwant to dive into sort of the wildestpart of experimentation this isdefinitely one of the places you canstart and it's really within reach allyou have to do is download one of thestarter projects for it and you can kindof see right in the prompting here's howyou kick start that process ofof iterationall right so I know that was super highlevel uh I hope it was useful uh it's Ithink from the field from the bottoms upwhat we're seeing and what people arebuilding kind of the this high levelcategories of apps that people aremaking all of these apps are apps thatare within reach to everybody which isreally really exciting uh and there's Isuggest Twitter is a great place to hangout and uh build things uh there's a lotof AI builders on Twitter uh publishingand if I think we've got a coupleminutes before Pizza is arriving maybe10 minutes keep on going oh so ifthere's any questions why don't we uhkick it to that because I'm sure there'ssome uh uh questions that you all have Iguess I ended a little early yes40 of the timeyeah do you have any like actualrecommendations uhofso the question was how approximatelyhow do you manage the hallucinationproblem like if you give it a physicslecture and you ask it a question on theone hand it appears to be answering youcorrectly on the other hand it appearsto be wrong to an expert's eye 40 of thetime 70 of the time 10 of the time it'sa huge problem and then what are someways as developers practically you canuse to mitigate that I'll give an answerstill you may have some specific thingstoo so one high level answer is the samething that makes these things capable ofsynthesizing information is part of thereason why it hallucinates for you soit's hard to have your cake and eat ittoo to a certain extent so this is partof the game in fact humans do it toolike people talk about you know uh justfolks who kind of are too aggressive intheir assumptions about knowledge Ican't remember the name for thatphenomenon where you'll just say stuffright so we do it tooum some things you can do are kind of arange of activities depending on howmuch money you're willing to spend howmuch technical expertise you have thatcan range from fine-tuning a model topractically so I'm in the applied worldso I'm very much in a world of duct tapeand sort of how developers get stuffdone so some of the answers I'll giveyou are sort of very duct tape answersgiving it examples tends to work foracute things if it's behaving in wildways the more examples you give it uhthe better that's not going to solve thedomain of all of physics so for thedomain of all the physics I'm gonna I'mgonna bail and give it to you because Ithink you are far more equipped than meto speak on that sure so the modeldoesn't have a ground truth it doesn'tknow anything any sense of meaning thatis derived from the training process ispurely out of differentiation one wordis not another word words are not usedin the same context it understandseverything only through examples giventhrough language it's like someone wholearned English or how to speak but theygrew up in a featureless gray roomthey've never seen the outside worldthey have nothing to rest on that tellsthem something is true and something isnot trueso from the models perspectiveeverything that it says it's true it'strying its best to give you the bestanswer possible and if it lying a littlebit or conflating two different topicsis the best way to achieve that then itwill decide to do soit's a part of the architecture we can'tget around it there are a number ofcheap tricks that surprisingly get it toconfabulate or hallucinate less one ofthem includes recently there was a paperthat's a little funny if you get it toprepend to its answermy best guess is that will actuallyimprove or reduce hallucinations byabout 80 percent so clearly it has somesense that some things are true andother things are not but we're not quitesure what that is to add on to what Tedwas saying a few cheap things you can doinclude letting it Google or Bing as inBing chat what they're doing it citesthis information asking it to make sureits own response is good if you've everhad shotgpt generate a programthere's some kind of problem and you askChachi PT I think there's a mistakeoften it'll locate the mistake itselfwhy it didn't produce the right answerat the very beginning we're still notsure but we're moving in the directionof reducing hallucinations now withrespect to physics you're going to haveto give it an external database to reston because internallyfor really really domain specificknowledge it's not going tobe as deterministic as one would likethese things work in continuous spacesthese thingsthey don't know what is wrong what istrue and as a result we have to give itto us so everything that Ted demo todayis reallystriving at reducing hallucinationsactually really and giving it moreabilities I hope that answers yourquestion and one of the ways too that Imean I'm a simple guy like I I tend tothink that all of the world tends to bejust a few things repeated over and overagain and we have human systems for thisyou know in a team like companies workare a team playing Sport and we're notright all the time even when we aspireto be and so we have uh systems thatwe've developed as humans to deal withthings that may be wrong so you knowhuman number one proposes an answerhuman number two checks their work humannumber three provides the follow finalsign off this is really common anybodywho's worked in a company has seen thisin practice the interesting thing aboutthe state of software right now we tendto be in this mode in which we're justtalking to GPT as one entity but once westart thinking in terms of teams so tospeak where each team member is its ownagent with its own set of objectives andskills I suspect we're going to startseeing a programming model in which theway to solve this might not necessarilybe make a sing single brain smarter butinstead B draw upon the collectiveintelligence of multiple software agentseach playing a role and and I think thatthat would certainly follow the humanpattern of how we deal with this to givean analogy space shuttles things that gointo space spacecraft they have to begood if they're not good people die theyhave like no no margin for error at alland as a result we over engineer inthose systems most spacecraft have threecomputers and they all have to agree inunison on a particular step to goforward if one does not agree then theyrecalculate they recalculate theyrecalculate until they arrive atsomething the good thing is thathallucinations are generally not asystemic problem in terms of itsknowledge it's often a one-off the modelsomething tripped it up and it justproduced a hallucination in that oneinstance so if there's three modelsworking in unison just instead of sayingthat will generally speaking improveyour success yeahyeah so hey Siriwhat's the mechanism by which thatinfluences thissure I'm going to give you what might bean unsatisfying answer which is it tendsto work but I think we know why it tendsto work and again it's because theselanguage models approximate how we talkto each other so if I were to say to youhey help me out I need you to mockinterview me that's a direct statement Ican make that kicks you into a certainmode of interaction or if I say to youhelp me out I'm trying to apologize tomy wife she's really mad at me can yourole play with me that kicks you intoanother mode of interaction and and soit's really just a shorthand that peoplehave found to kick the agent in to kickthe llm into a certain mode ofinteraction that tends to work in theway that I as a software developer amhoping it would workand to really quickly add on to thatum being in the digital Humanities thatI am I like to think of it as anarrative a narrative will have a fewdifferent characters talking to eachother their roles are clearly definedtwo people are not the samethis interaction with GPT it assumes thepersonality it can simulatepersonalities it itself is not cautiousin any way but it can certainlypredict what a conscious being wouldreact like in a particular situation sowe when we're going u r x it is drawingup that personality and talking asthough it is that person because it isit is like completing a transcript orcompleting a story in which thatcharacter is present and interacting andis activeso yeahI think we've got about five minutesuntil the the pizza outsideanywaysyes sir all right yes sobut um it's been fun playing with thisand I understand the sort of word byword generation and the sort of vibe thefeeling of it you know the narrativesome of my friends and I have triedgivingproblems like things from the LSAT forexample and it doesn't work like and I'mjust wondering why that would be so itlike it will generate answers that soundvery plausible rhetorically like giventhis condition yeah why why but it'lloften like even contradict itself in itsanswers but it's almost never correct soI was wonderingwhat why that would be like it justcan't reason it can't like think andlike can you would we get to a placewhereyou can I mean not you know what I meanI don't even think like it's conscious Imean like have thoughts you want to talkabout reactso gpt4 one when gpt4 released back inMarch I think it was it was passing LSATit was yeah yes yes it just passed as Iunderstand itbecause that's one of the weird thingsis thatyeahif you pay for job GPT they give youaccess to the better model andone of the interesting things with it isprompting it's so finicky if you it'svery sensitive to the way that youprompt there were earlier on when gpt3came out some people were going look Ican pass literacy tests or no it can'tpass literacy tests and then people whoare pro or anti-gpt would be like Imodified The Prompt a little bitsuddenly it can or suddenly it can'tthese things are not cautious theirability to reason is like an aliensthey're not us they don't think likepeople they're not human but theycertainly are capable of passing somethings empirically which demonstratessome sort of rationale or logic withinthe model but we're still slowlyfiguring out like a prompt Whispererwhat exactly the right approach ishave you seeninstances where it's directly createsome sort of business value inAI appsyeah I mean we host companies on top ofus who that's their their primaryproduct uh the the value that it adds islike any company I mean it's you knowwhat is the Y combinator mono makesomething people want I mean I wouldn'tthink of this as GPT inherently providesvalue for you as a builder like that'stheir product that's Open the Eyesproduct you pay chat GPT for prioritizedaccess where your product might be ishow you take that and combine it withyour data somebody else's data somedomain knowledge some interface uh thatthen helps apply it to something it istwo things are both true there are a lotof experiments going on right now uhboth for fun and people trying to figureout where the economic value is but butfolks are also spinning up companiesthat are 100 supported by applying thisto dataI I I think that it is likely that todaywe call this GPT and today we call thesellms and tomorrow it will just slideinto The Ether I mean imagine what theimagine what the progression is going tobe today there's one of these that thepeople are primarily playing withthere's many of them that exist but onepeople are primarily bidding atoptomorrow we can expect that there willbe many of them and the day after thatwe can expect they're going to be on ourphones and they're not even going to beconnected to the internet and for thatreason I think that like today we don'tcall our software microprocessor toolsor microprocessor apps like theprocessor just exists I think that oneuseful model five years out ten yearsout is to even if it's onlymetaphorically true and not literallytrue I I think it's useful to think ofthis as a second processor we had thisbefore with uh with with floating pointco-processors and Graphics co-processorsalready as recently as the 90s whereit's useful to think of the trajectoryof this as just another thing thatcomputers to do can do and it will beincorporated into absolutely everythinghence the term Foundation model whichalso crops upI'm sorry sopizza is ready one more question maybeone more and then uh then we'll breakfor some foodin the glasses right thereI was just being told we need to get twomore so yeahit's hard to get it to do that reliablyit's incredibly useful to get it to doreliably so some tricks you can use areyou can give it examplesyou can just ask it directlyuh those are two common tricksum and and look at the prompts thatothers have used to work I mean there'sa lot of art to finding the right promptright now a lot of it is Magicincantation another thing you can do ispost process it so that you can do somechecking and you can have a happy pathin which it's a one shot and you getyour answer and then a sad path in whichmaybe you fall back on other prompts sothen you're going for the diversity ofapproach where it's it's fast by defaultit's slow but ultimately converging uponhigher likelihood of success if it failsand then something that I'm sure we'llsee people do later on is is fine tuneinstruction tuning style models whichare more likely to respond with thecomputer partsable uh output so I guessone one last question so sure um so theone you talked a couple of thingsone is as you talk about domainexpertiseyou're encoding a bunch of domainexpertise in terms of the prompts thatyou'redoing what is that where do thosepromptsand is therethat's a great question so the questionwas and I apologize I just realized wehaven't been repeating all the questionsfor the the YouTube listeners so I'msorry for the folks on YouTube if youweren't able to hear some of thequestions uh the question was what arethe Privacy implications of some ofthese prompts if one of the messages isso much depends upon your prompt and thefine-tuning of this prompt what doesthat mean with respect to my IP maybethe prompt is my businessI can't offer you the exact answer but Ican paint for you what approximately thelandscape looks like so in all ofsoftware and so too with AI what we seeis they're the SAS companies whereyou're using somebody else's API andyou're trusting that their terms inservice will be upheld there's the setof companies in which they provide amodel for hosting on one of the bigcloud providers and this is a version ofthe same thing but I think with slightlydifferent mechanics this tends to bethought of as the Enterprise version ofsoftware and by and large the industryhas moved over the past 20 years fromrunning my own servers to trusting thatMicrosoft or Amazon or Google can runservers for me and they say it's myprivate server even though I knowthey're running it and I'm okay withthat and you're gonna you've alreadystarted to see that Amazon with huggingface Microsoft with open AI Google 2with their own version of Bard are goingto do these where you'll have the SASversion and then you'll also have theprivate VPC version and then there's athird version that I think we haven'tyet seen practically emerge but thiswould be the the maximalist I want tomake sure my IP is maximally safeversion of events in which you arerunning your own machines you arerunning your own models and then thequestion is is the open source and orprivately available version of the modelas good as the publicly hosted one anddoes that matter to me and the answer isright now realistically it probablymatters a lot in the fullness of timeyou can think of any one particular taskyou need to achieve as requiring somefixed point of intelligence to achieveand so over time what we'll see is theprivately obtainable versions of thesemodels will cross that threshold andwith respect to that one task yeah sureuse the open source version run it onyour own machine but we'll also see theSAS intelligence get smarter it'llprobably stay ahead and then you'requestion is which one do I care moreabout do I want like the betteraggregate intelligence or is my tasksomewhat fixed point and I can just usethe open source available one for whichI know it'll perform well enough becauseit's crossed the threshold so to answeryour question specifically yes uh youmight be glad to know if childcptrecently updated their privacy policy tonot use prompts for the training processbut up until now everything went backinto the bin to be trained on againand that's just a fact so I think Pizzais now Pizza time let's go get somepizza I hope this was useful uh we'll bearoundtwo days questions