Google AI Documents Leak about “Google and OpenAI”

so there's some interesting news outtoday there's a leaked Google documentthat talks about the state of AI andwhere Google is in terms of AI and itsprogression called we have no moat andneither does open Ai and it's got somepretty big claimsum it was leaked by a Google employeeit's from a researcher inside of Googleso it starts out saying we've done a lotof looking over our shoulders at open AIwho will cross the next Milestone whatwill the next move be so let's talkabout the fact that Google was seen asthe number one AI company for a longtime then openai kind of comes out ofnowhere and maybe I would even go as faras say dominates the AI playing field ofChad GPT all their other software andnow it's front and center in the talkabout Ai and where we're heading then itcontinues but the uncomfortable truth iswe aren't positioned to win this armsrace and neither is open AI while we'vebeen squabbling a third faction has hasbeen quietly eating our lunch I'mtalking of course about open sourceplainly put they are lapping us so thisthis document goes on to say that a lotof the big problems that Google wasworking on they couldn't quite figureout was solved kind of quickly when someof these models were just released tothe public people are running Foundationmodels on a pixel 6. people can finetune a personalized AI on your laptop inone evening their entire website's fullof art models with no restrictionswhatsoever text is not far behind andthe current multimodal science QA wastrained in one hour this person goes onto say while our model still hold aslight Edge in terms of quality the Gapis closing astonishingly quickly opensource models are faster morecustomizable more private andpound-for-pound more capable we have nosecret sauce there's nothing there's nomoat that protects them from everybodyelse sort of coming in and having thesame software the same capabilities Etcwell interesting point was this giantmodels are slowing us down in the longrun the best models are the ones whichcan be iterated upon quickly this isbasically how quickly an open sourcemodel is doing 90 percent of what youknow ninety percent of what Chad gbt isdoing the kuna I'm gonna assume it'spronounced vicuna an open sourcedchatbot and pressing gpt4 with 90 ChadGPT quality and then then it goes on toshow the side by side comparison andwhere the kuna is next to Chad gbt andBard Etc so and then the researcher askswhat happened so basicallymeta or Facebook as they used to becalled uh had this AI model called llamaand it was leaked to the public so thismodel it had no instruction orconversation tuning and had no rlfhwhich means um reinforcement learningwith human feedback nonetheless thecommunity immediately understood thesignificance of what they have beengiven and tremendous outpouring ofinnovation followed and within just daysbetween major developments and they'regoing to have a timeline at the end ofthis article that I'll show you butbasically barely a month later therewere huge variants with instructiontuning quantization quality improvementshuman vowels multimodality rlfh and manyof which build on top of each otherbasically what this means is as soon asthis thing got out there it was sort ofthe global Community started working onit and the Innovations just came muchmuch faster than they would if theywould have stayed behind closed walls ata big Tech firm now he's saying why wecould have seen it coming and and inmany ways this shouldn't be a surpriseto anyone so basically he's comparingthis to stable diffusion where low costpublic involvement was enabled by avastly cheaper mechanism mechanism forfine-tuning called low rank adaptationor Laura and in both cases access to asufficiently high quality model kickedoff a flurry of ideas and iterationsfrom individuals and institutions aroundthe world in both cases this quicklyoutpaced the larger players thesecontributions were pivotal in the imagegeneration space setting stablediffusion on a different path from Dollyso what this is saying is that thesoftware the AI models that get put outto the public where everyone contributesthose tend to dominate versus you knowthese closed models in terms of culturalimpactInnovation product Integrationsmarketplaces user interfaces Etcand I say Dali while it was impressiveat first comparing it to something likemid-journey I gotta say like stablediffusion majority there they seem muchfarther advanced than Dolly at thispoint so whether the same thing willhappen for llms remains to be seen Butthe broad structural elements are thesame many of these projects are savingtime by training on small highly curateddata sets this suggests that there'ssome flexibility in data scaling lawsand of course he says that directlycompeting with open source is a losingproposition then he goes on a bit toexplain that basically Google can'tcompete with this they can't just lockpeople up for using it people can use itfor personal use they will understand itbetter like the illegal cover affordedby personal use and the impracticalityof Prosecuting individuals means thatindividuals are getting access to theseTechnologies while they are hot beingyour own customer means you youunderstand the use case so basicallylarge like monolithic institutions canbe as good as basically the whole Worldcontributing and working quickly andsharing what what they've learned and hesays paradoxically the one clear winnerin all this has met him because theleaked model was theirs they haveeffectively garnered an entire planetsworth of Free Labor since most opensource Innovation is happening on top oftheir architecture there's nothingstopping them from directlyincorporating it into their products sowhat he's saying is that Facebookactually this was kind of a big win forthem even though it was a massive scopethe only time that they couldn't dosomething genius is when they screw itup and it just happens I mean he's notsaying that I'm saying that but yeahGoogle and openai have both gravitateddefensively towards release patternsthat allow them to retain tight controlover how their models are used but thiscontrol is a fiction and he says wecannot hope to both Drive Innovation andcontrol itand he's saying that in the end open AIdoesn't matter they were such a bigthing that happened but in the endthey're just not gonna matter no one'sgonna have a moat and on Google notopening eye nobody open sourceAlternatives can and will eventuallyEclipse them and unless they changetheir stance in this respect at least wecan make the first move meaning thattheir opening eye overcame them and isahead of them in some ways but they'resaying if they see where the puck isgoing they can start off start movingtowards that spot first before openingeye but becoming more open and opensourceand so this is the timeline so February24th 2023 llama is launched uh March 3rd2023 the inevitable happens which isllamas leak to the public I remember theday that happened that was huge andpeople were kind of going nuts over thisthing and so then a little over a weeklater so March 12 language models on atoasterum a week a little over a week laterartem gets the model working on aRaspberry Pi now the model works tooslowly but it sort of sets the stage foran onslaught of medication efforts greatuse of the word onslaught I love it thenext day Stanford releases alpacathey're able to do fine tuning withinhours on a single I think that's anNvidia card so they're able to dotraining on a single RTX 4090 sosuddenly anyone could fine-tune themodel to do anything kicking off a raceto the bottom on low budget fine-tuningprojects papers proudly describe theirtotal spend of a few hundred dollarsthis is why one of the reasons why AI isso exciting and so mind-blowing to mebecause the the things that it could dopotentially is unlimited and the cost todo it seems to be so much cheaper thanwe can even realize and it's gettingcheaper still it's like the mostpowerful thing at and ease it's it'scrazy and then March 19 2023 we havethat vicuna I don't know if I'm sayingthat right but it's uh catching up withBard and Chad GPT and then there's somesort of an open source gpt3 and thesemodels are trained from scratch meaningthe community is no longer dependent onllama they took this leak of llama andthey tinkered with it until theybasically replicated now they have theirown thing that is not sort of dependenton Facebook or meta in any way it's outthey they basically copy and pasted itout into the wild so I'm going to postthis article on natural20.com that'snatural2020.com go check it out my name is WesRoth thank you for watching