Emerging AI: This Week’s Mind-Boggling Advances!

there's been a lot of cool Ai and TechInnovation that's come out over the lastseveral weeks and I want to break downsome of the coolest stuff that I've comeacross over the last few days startingwith this really cool video that Mrboofy made of me in this same officehere he used a tool called warpdiffusion which is still somethingthat's beyond me but I kind of want tolearn how to use it once I figure outhow to do this myself I might make atutorial on it but it's a little bitmore complex than most of the tutorialsI've made to date but I just thoughtthis was a really really amazing videoso I wanted to share it at the beginningof this video shout out to Mr boofy forcreating this everybody should go followhim on Twitter he creates all sorts ofcool videos in this same style here I'mgoing to start off with some coolresearch and stuff that's in the worksand then I'm going to wrap it up withsome cool tools that you can use rightnow today starting with this researchcalled Mind video now in the past I'veshown you research where they'rescanning people's brains with MRIs whileshowing them images and then using aprocess to actually convert what theMRIs are seeing back into imagesessentially allowing us to read people'sminds and see the images that they'reseeing well now they're starting to beable to do that with video you couldread about this latest research over atmind-video.com but basically whatthey're doing is they're putting peopleinto an MRI machine they're showing themvideos while they're in the MRI machineusing their brain encoding process andthen decoding what the brain is seeinghere's an example video These are framesfrom the video of a cat here and thenwhat was reconstructed was this videofrom the brain now they say what this isdoing is semantically reconstructing thevideo meaning that it's getting theessence of what they're seeing it'sgetting the idea but it's not showingexactly what they're seeing so here'ssome other examples in this one thevideo on the left was a video of somefish that they were showing them thevideo on the right is what the brainscans produced which you could tell iskind of an underwater fish sort of videohere's another one where the one on theleft shows a video of like a jelly fishand on the right it's a fish you cantell it's underwater in the ocean itsort of got the idea here's another onethey showed him a video of some horsesyou can see on the right some horses onthe left a turtle it translated moreinto like fish but it's still theunderwater animal scene they showed himthis video on the left of a man walkingwith his dog I believe with their braindecoding decoded was this video herewhere it looks more like a couplewalking with a dog but kind of going inthe different direction here's a videoclip of a man and woman side by side andthe sort of semantic response was thiswhere it's a man talking to a womanhere's another one with a scene of a cardriving through a little toll booth andthen a river and then the result wassort of a Mountain Road we got a littlevideo of a bird here what it saw wasthis bird and then finally a video of acrowd walking and on the right a videoof a crowd walking again the pastresearch was mostly doing image to imagewhere they were showing them imagesscanning their brain and then gettingsimilar images back now they're startingto be capable of doing this with videoit's not perfect it's still semanticallycorrect it's getting this sort ofessence the the idea of the video thatthey're seeing but it's still notperfect but this is brand new researchthis is very very early so exciting tosee where this is going again I don'treally think we have to worry about youknow governments using this technologyto read our minds or anything yetbecause it still requires people beingput inside of an MRI machine with theirbrains being scanned and I don't believethat governments are anywhere close tobeing able to do this at a large scaleon a lot of people so I think ourthoughts are still safer right now nowhere's an interesting piece of tech thatI came across on Twitter from BjornCarmen I'm sorry if I mispronounced yourname but this is basically a camera withno lens here take a peek so we adjustsome dials on the camera you can see heholds it up he adjusts The Meters thatit's taking the picture of and he snapsa photo and it analyzes the scene andthen essentially writes a prompt andthen generates an image based on theprompto that's the original scene thereand that's the image that it took basedon creating a prompt and then generatingthe image off the prompt now while Ithink this technology is reallyinteresting and cool to see I'm not ahundred percent sure I understand thePractical use of it especially if Icould get on like Google Maps find alocation and get a similar result and ifI'm actually in a specific location andI have a camera with me I'd probablywant to take a picture of what I'mactually seeing but let's be honest itis a pretty interesting and originalidea I'd love to hear what people thinkmight be some interesting use cases fortechnology like this so earlier thisweek on Sunday Jensen Huang the CEO ofNvidia gave his first keynotepresentation in four years and I'm goingto break down more of that keynotepresentation in Friday's news breakdownvideo but one thing that he showed offthat I thought was really cool was thisvideo where a real human goes andinteracts with somebody inside of avideo game and the person in the VideoGame responds real time and theirresponse is generated by AI it's notpre-written dialogue so this video I dohave shared on my Twitter if you want towatch the full thing here make sureyou're following me over at Mr e flowbut let's take a peek at this video hereforeign[Music]how are youunfortunately not so goodhow comeI'm worried about the crime around hereit's gotten bad latelymy Ramen Shop got caught in thecrossfirecan I helpif you want to do something about this Ihave heard rumors that the powerfulcrime Lord Kumon Aoki is causing allsorts of Chaos in the city he may be theroot of this violenceI'll talk to him where can I find himI have heard he hangs out in theunderground fight clubs on the city'sEast Side try thereokay I'll goI'm careful Kai[Music]none of that conversation was scriptedwe gave this gen AI character abackstory his story about his Ramen Shopand the story of this game all you haveto do is go up and talk to thischaracter and because this character hasbeen infused with artificialintelligence and large language modelsit can interact with you you understandyour meaning in a really reasonable wayso like Jensen said that was allgenerated with AI I would have loved tosee them have a different conversationbecause the conversation they had inthat video looked like it could havebeen pre-scripted right he walked intosomebody's Ramen Shop asked how he wasdoing the guy said my Ramen Shop's beencaught in the crossfire guy asked wellwho do you think did it he gave him someinformation and it felt like it couldhave been pre-written dialogue it wouldhave been really interesting for the enduser to walk into the ramen shop andhave a little bit more of an in-depthdiscussion before diving into what heneeded to do next because otherwise it'ssort of not really too different fromhaving pre-written dialogue now onething that's really cool is one of thecreators of this technology the founderof convey reached out to me and we'regonna have a chat and he's gonna give mea little bit deeper of a walk through ofhow this technology actually works soexpect a follow-up video where we diveeven deeper on how this technology worksand where it's headed now this idea ofbeing able to have conversations withNPCs and video games and have theconversation be generated by AI isn'tsuper new in fact my buddy Matt vidprohere did a video about it two months agowhere he streamed playing an AI game thebig difference is the timing it's thelatency right when we just watched thatin video demo the person asked aquestion and the response came nearlyimmediately watch how the existing gamesthat are on the market that do this kindof thing work right now just payattention to the pause in there listenwhat's your name can I call yousweetheart if you don't mind do you havea family do you have kids and mostimportantly do you play Among UsI appreciate your attempts at small talkbut let's keep our focus on thisinvestigation my name is SergeantShannon and as for your questions I amnot comfortable sharing personalinformation with you it is for Among UsI'm afraid I haven't had a chance toplay it yet what are you doing thisweekend I gotta askdetective as much as I'd love to chatabout weekend plans we have morepressing matters at hand let's stick todiscussing relevant details about thiscase and work together to ensure PublicSafety so you can see there's quite abit of delay between Matt asking thequestion and the response this video isactually hilarious Matt asks a lot ofreally funny questions so I'll make sureit's linked up Below in the comments soyou can watch his full live stream andSkip around uh some of the stuff hetries to get the AI to say is prettyhilarious but uh I won't spoil it youshould check out his video continuing onwith the theme of gaming this is brandnew research called Voyager that'sactually training the computer to playMinecraft it could do things like mineamethyst gather Cactus build a basefight Enderman build a house mine goldbuild Nether Portal hunt pigs swim andbasically do all of the tasks thatsomebody might need to do in Minecraftnow they obviously don't know that rulenumber one of Minecraft is that younever dig straight down never digstraight down but this is a learningcomputer now the idea of a bot thatlearn to play Minecraft on this surfacemay not sound that impressive but thetechnology and what they builtunderneath is actually quite cool sobasically what this model is trying todo is it's trying to become the bestMinecraft player in the world and theway it's doing that is it's by going andlearning how to do different skills andonce it figures out how to do a skill itsaves that skill into its own internalmemory so that it can remember how to dothat again later and then moves on tolearning another skill and then it savesthat skill as code inside of itsinternal memory and it repeats thisprocess and repeats this process untilit gets better and better and better atMinecraft so it's a learning bot that'sessentially teaching itself to getbetter at Minecraft once it gets good atone skill saves that skill inside of itscode memory and then continues to stackskills on top of it while the use casethat they're demonstrating this with isMinecraft the underlining technology ofthis is really really really powerfulwhen you think about it is aself-learning bot that will learn a newskill save that skill to memory move onlearn the next skill and continue tostack skills on top of itself until it'sthe best at the final larger skill thatis trying to accomplish so here's whythey specifically chose Minecraft unlikemost other games studied in AI Minecraftdoes not impose a predefined end goal orfixed storyline but rather provides aunique playground with EndlessPossibilities you're not trying tonecessarily beat the game you're justtrying to get better and better andbetter and explore and build and justfigure out new tasks to do in the gamean effective lifelong learning agentshould have similar capabilities ashuman players so it will proposesuitable tasks based on its currentskill level and World State learn toharvest sand and Cactus before iron ifit finds itself in a desert rather thana forest it will then refine skillsbased on environment feedback and commitmastered skills to memory or featurereuse in similar situations for examplefighting zombies is is similar tofighting spiders and continually explorethe world and seek out new tasks in aself-driven manner and it's sort of thisperpetuating cycle that will continue toLoop seek out new tasks learn how to dothose tasks commit them to Memorycontinue to seek out new tasks learn howto do those commit them to Memory repeatthe process until it becomes the bestMinecraft player in the world when youreally understand what they're trying toaccomplish and you understand that it'snot just about Minecraft you can see howthis can be powerful in other areas it'sessentially an autonomous agent butcreated for use in a Minecraft world andto top it all off they've actually opensourced the code and put it on GitHub soyou can install it on your own computerand train your own Minecraft Agent toget better and better if you'd like sohere's something else that's in theworks that was sent to me by the peopleover at in video to check out now we'veseen a lot of text to video alreadywhere it will generate short clips wherethey're you know maybe three fiveseconds all the way up to 15 secondswith stuff like gen 1 but even thatrequires a video input well what Nvidiais doing is creating longer form text tovideo where you can type in a promptwhere you expect a longer video and theydid send over this demo video so let'stake a peek at it a cinematic videoabout a monk with super powers it'sgenerating music generating soundeffects and this is the video itgenerated I'm not playing the musicbecause I don't actually know thelicensing on it but you can see there'sthis video of a monk another Monkmeditating now I don't believe this isAI generated video I think maybe whatit's doing is it's creating thenarrative for the video and then findingstock video to go along with it so Idon't believe what's actually beinggenerated is AI generated it's justbeing AI sourced video so to speak butthe people at it in video I am going tohave a conversation with them soon andthey're going to clarify this for me andwe'll be demoing this more but whatwe're getting to a point now where youcan enter a prompt for a full longervideo and it will write the script findthe music generate sound effects pull inall the video clips that you need andgenerate that video now it'll get reallyexciting when it starts actuallygenerating that video purely with AI butright now it's still pretty damn coolthat it's going to be able to generateit using stock footage that it findsfrom around the internet to go alongwith the narrative that's on the screenso I'm excited to see this either wayall right now I'm going to move intosome of the stuff that you can actuallyplay with right now if you want tostarting with this cool video that Icame across for a tool called ecootwhich I believe is the French word forlisten it's a tool for online interviewsif somebody's interviewing you for a jobit will actually listen to the questionsthat they're asking during that jobinterview and then feed you the bestresponse that you can give for Thatinterview question now this isn't a realinterview here this is sort of a demointerview but check this out what aboutyour development areas what do you haveidentity fight as your greatest andbiggest Improvement areas you can seeover on the right you can see what thespeaker is asking even to the right ofthat you can see the recommendedresponse to the question and what haveyou done to improve them so far I wouldsay my greatest Development Area is mycommunication skills I work on improvingmy ability to clearly convey my thoughtsand ideas to others I've been working onimproving by practicing my activelistening and so you can see that as theinterviewer asks the questions itlistens to the interviewer and thengives you the best response to thatinterview question which is kind offunny because doesn't that sort of puteverybody on a Level Playing Field whenit comes to doing an online jobinterview as this sort of Technologygets more prolific I really doubtcompanies are going to be able to dointerviews in this fashion becausepretty much anybody can be eloquent andwell thought out with their responses tointerview questions again this is opensource you can actually install it ifyou're on Windows it's says it's onlybeen tested on windows so far doesn'tmean you can't use it on a Mac they justhaven't tested it on a Mac yet and allyou got to do is follow theseinstructions to get it set up I'll makesure this GitHub is linked below thevideo in the description so if you wantto check this out and play around withthis tool yourself it will be linked upbelow alright so this is one where theyreached out to me and sent me this demoand I was blown away by it so we've seentools like did right where you canupload an audio file and then upload apicture and it'll make that picture sortof talk like the audio file I've done ahandful of videos demonstrating it andhave all sorts of use cases for itimagine that but instead of startingwith a picture you're starting with avideo so for example here's a video fromHarry Potter of Voldemort walkingforward and then looking into the cameraand then let's say I want this Boratvoice on the video he's my neighborI could click generate Lip Sync It'sgoing to process this it's fairly quickto process he's my neighborhe is failing my ass Falls I get awindow from a glass you must get thewindow from a glass I get a step youmust get the step I get the clock radiohe cannot afford great success so youcan see whoever's on the screen in thatmoment it actually moves their lip andlip syncs it to the audio that youuploaded and you can upload your ownvideos and you can upload your own audiofile and do it with any clip so forexample let's go to 11 Labs real quickI'm going to use my cloned voice hereI'm going to say my name is Matt wolfandum reporting for Duty sir let's generatethat my name is Matt wolf and I'mreporting for Duty sir let's go aheadand download this clip that we just madeI've got this clip from Forrest Gumplet's add our own audio I'll clickupload audio and grab this 11 Labs audioI just made and let's generate lip syncmy name is Matt wolf and I'm reportingfor Duty sir and now I just made myvoice over Forrest Gump and his lipsmove to what I just said my name is Mattwolf and I'm reporting for Duty sir myname is Matt wolf and I'm reporting forDuty sir my name is Matt wolf and I'mreporting for Duty sir so did is greatfor animating still images but thislalamu tool this is what we're going tostart using to lip sync over existingvideos so my prediction if you're onTwitter over the next several monthsyou're probably going to see a lot ofthese types of videos circulating prettysoon all right finally I want to wrap upwith talking about Adobe photoshop'sgenerative fill just one more time Iknow I've talked about it in a coupledifferent videos before I made a fullvideo about it the other day showing offsome of my own experiments with it plussome stuff other people did and someamazing commenters on this post sharedsome tips to actually use generativefill even better for instance don't usethe magic wand to select the gender offill area because you want a bit of anoverlap with the original image I thinkwhen you use generative fill you have tomake a selection where it's slightlyoverlapping the original pick so thatsame comment was shared by a handful ofpeople so I just wanted to touch backinto Adobe photoshop's generative fillone last time and try a couple morequick tips that I've seen withgenerative Phil so if you remember I wasplaying around with this image the otherday and I tried to add some legs to himbut instead it put a countertop so I'mgoing to zoom out a little bit I'm gonnago to my crop tool and I'm going to goahead and extend this down quite a bithere now I've got a whole bunch of extraarea below his body let me zoom out alittle bit more just so we can see thewhole thing on screen and I'm going togo ahead and grab the selection toolthis time select this whole area butthen make sure it overlaps a little bitwith what's above it and now let's trythe generative fill and would you lookat that it actually gave him crazymonster hands but it did give him legsso let's see some of the other optionsit gave there's that one looks muchbetter with a closed fist for sure andthen there's that one so pretty dangimpressive now another cool trick thatI've seen was taking two differentimages and then using generative fill tofill in the gap for those images solet's go ahead and create a new imagehere and I'm going to pull in twoseparate images I'm going to pull inthis image that's supposed to be me likeas a wizard I'm going to make it alittle bit smaller let's put it downactually no let's put it right here andthen I'll pull in this image over herethat's got these two wolves in it theylook like they could be sort of similarso what I'm going to do is I'm going tomerge these two images together so thatit's using both of them for contextlet's merge the layers now it's just onelayer here I'm going to use my magicwand tool to select the background butthen I'm going to bring it in a littlebit so that it overlaps with the imagesa tiny bit so if I go to select and thenmodify and then expand let's expand itby 10 pixels and you can see now itoverlaps into the images a little bitand let's just go ahead and dogenerative fill and see if it figuresout how to connect these two imagestogether so there's one you candefinitely see a hard seam down herewhere the two images kind of split butit gave me more body it kind of tried tocombine the trees let's see anotheralternative this one's not too badeither less of a seam it kind of blurredthe lines there and then this one'spretty good as well the color palettesare slightly different so you could tellthat it started as two different imagesthis is a little bit more hazy in thebackground that to me is really reallycool this image was generated in stablediffusion this image I believe was madein mid-journey I pulled both thoseimages in use generative fill and itkind of made them all look like oneimage now that's pretty mind-blowing tome one last little fun thing I've seenpeople doing as a meme on Twitter that Iwant to try myself is to expand an albumcover so let's go ahead and createanother New Image here I grabbed thisiconic album cover from you know my highschool college Years so let's go aheadand add that in there I'm going to usethe magic wand tool again to select andthen I'm going to modify the selectionand I'm going to expand it a little bit25 pixels so that it's seeing a littlebit more of what's going on in the imagehere and then I'm going to usegenerative fill to have it generate whatit thinks is the rest of the album coverthere's one iteration of it here'sanother and here's another one can'treally tell what's going on in reallyany of them let's go ahead and generateagain and see if it comes up withsomething even funnier here's one whereshe's in the water I don't know what'sgoing on with that one I don't knowwhat's going on behind it but it lookscool nonetheless wanted to make thatlittle update to photoshop generativefill try some of the stuff that you guyswere suggesting in the comments try thismeme generation that I've been seeing onTwitter and have a little bit more funwith it because generative fill isreally impressive and I think in my lastvideo where I talked about it I kind ofundersold it a little bit it is reallyfun and really cool and that's all I gotfor you today I have a couple requestsof you if you enjoyed this video followme over on Twitter I am Mr e flow onTwitter I share AI news almost as ithappens I share some of the coolest AItools that I come across as I comeacross them I make a lot of extra videosthat are only available on Twitter thatI don't share on YouTube I Like to makesuper edits of longer videos where Itake two three hour long videos and editthem down into like 10 12 15 minutevideos that are just the tldr of thevideo I share all sorts of cool stuff onTwitter that it doesn't make it to myYouTube channel so if you're notfollowing me on Twitter follow me overat Mr e flow and as always if you likenerding out about all of this AI stuffand you want to find all the latestgreatest tools you want to keep in touchwith all the latest AI news in additionto my Twitter I also keep this Futuretools website up to date on a dailybasis if you just want a once a weekbreakdown of what's going on in theworld of AI join the free newsletter andevery Friday I'll send you the tldrwhich is my five favorite tools ahandful of news articles a handful ofYouTube videos and one cool way to makemoney with AI all you got to do is go tofuturetools.io and click join the freenewsletter and I'll start hooking you upthis Friday and final thing the lastcall to action I know if you're in themarketing world this is way too manythings to ask of a viewer but maybe givethis video a thumbs up if you liked itand subscribe to this channel if youhaven't already I really reallyappreciate it thanks for nerding outwith me see you guys the next one bye[Music]thank you[Music]