","posterPhotoId":null,"w":null,"h":null,"caption":{"text":{"className":"TextEmpty"},"credit":{"className":"TextEmpty"},"className":"PageCaption"},"className":"PageBlockEmbed"},{"text":{"text":"He says:","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"text":{"texts":[{"text":"If you allow any sufficiently wide-ranging access to your AI model, even by paid API, you're giving away your business crown jewels to competitors that can then nearly-clone your model without all the hard work you did to build up your own fine-tuning dataset. …\n\n","className":"TextPlain"},{"text":{"text":{"text":"I'm not sure I can convey how much this is a brand new idiom of AI as a technology.","className":"TextPlain"},"className":"TextBold"},"className":"TextItalic"}],"className":"TextConcat"},"className":"TextItalic"},"caption":{"className":"TextEmpty"},"className":"PageBlockBlockquote"},{"text":{"text":"That’s right. It’s the equivalent of the A student having other students copy his or her exam answers. It’s a short-cut, and if 52K prompts is only $600, what if you scale it up 1000x, with 52 million prompts, and fine-tune a model for $600K? Doable for a startup or even a research lab. How far will this go? As far as researchers and companies can take it.","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"text":"While he mentions that model companies might try to ‘protect’ the IP of their LLM, it begs a question of how much they can protect when they themselves basically have scraped the entire internet and digested whole libraries of copyrighted information to produce the model in the first place. Even if they do create some proprietary way to lock their models down, that would likely make them less useful and amenable to use as an AI platform within a larger ecosystem. Catch-22.","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"texts":[{"text":"Further, any model that tries to close themselves off from copying ","className":"TextPlain"},{"text":{"text":{"text":"won’t prevent the process from being used","className":"TextPlain"},"className":"TextBold"},"className":"TextItalic"},{"text":", and it seems to be unreasonably effective and economical. There will be ways to sanitize. Full models won’t be that expensive to replicate: A LLaMA-65B equivalent model has the incremental compute cost of 1 million A100 GPU hours, plus the cost of massive 4 trillion word data collection. A huge effort, but far from a moonshot.","className":"TextPlain"}],"className":"TextConcat"},"className":"PageBlockParagraph"},{"text":{"texts":[{"text":"So ","className":"TextPlain"},{"text":{"text":"let’s think step by step","className":"TextPlain"},"url":"https://arxiv.org/abs/2205.11916","webpageId":"0","className":"TextUrl"},{"text":" here. (Sorry, that prompt is a play on the ","className":"TextPlain"},{"text":{"text":"paper on zero-shot reasoning","className":"TextPlain"},"url":"https://arxiv.org/abs/2205.11916","webpageId":"0","className":"TextUrl"},{"text":" that showed the unreasonable effectiveness of a particular prompt for LLMs.)","className":"TextPlain"}],"className":"TextConcat"},"className":"PageBlockParagraph"},{"items":[{"text":{"text":"We will see a rapid advance of models and we will see a sort of AI-capability design convergence, as models copy each others capabilities to improve (either directly or indirectly).","className":"TextPlain"},"className":"PageListItemText"},{"text":{"text":"In the AI space, we are seeing that fast-followers have effective and cheap strategies for staying close to leaders. (In business terms, leaders are leading innovators while fast-followers are those that emulate and keep up with the leader in short order.)","className":"TextPlain"},"className":"PageListItemText"},{"text":{"text":"It may be hard for leaders to differentiate on quality from the ‘fast-followers’ and thus hard to extract pricing premium on a sustainable basis.","className":"TextPlain"},"className":"PageListItemText"},{"text":{"text":"Thus, the pricing for best-in-class foundational model AI inference will go to marginal cost of computing the inference and API call. It will become a bit like the Intel / AMD CPU market: There is some premium for IP, but it remains a very competitive market pricing-wise.","className":"TextPlain"},"className":"PageListItemText"}],"className":"PageBlockList"},{"text":{"text":"The one consideration where this thinking may go wrong is that ecosystems can create a kind of lock-in effect, and the amount of data a leader generates through the use of their tool might give them a sustainable advantage if it helps them continually improve their model quality to keep ahead. In that scenario, there is a constant march of improvement by the leader(s) that makes it hard for others to keep up with.","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"text":"Eliezer concludes:","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"text":{"text":"“The AI companies that make profits will be ones that either have a competitive moat not based on the capabilities of their model, OR those which don't expose the underlying inputs and outputs of their model to customers, OR can successfully sue any competitor that engages in shoggoth mask cloning.”","className":"TextPlain"},"className":"TextItalic"},"caption":{"className":"TextEmpty"},"className":"PageBlockBlockquote"},{"text":{"text":"I tend to agree.","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"text":"The competitive moat that Microsoft / OpenAI ecosystem (or a Google / DeepMind / Anthropic ecosystem) can build is based on cheap compute, easy-to-use UI/UX, serving an effective, general, high-quality AI model (LLM) endpoint. The AI model endpoints for general use will be driven by the marginal cost of inference, with some premium for the unique and superior qualities of that model. The amount of that premium is to be determined, but these factors above suggest it will be capped.","className":"TextPlain"},"className":"PageBlockParagraph"},{"text":{"texts":[{"text":"So, the bottom-line on pricing: ","className":"TextPlain"},{"text":{"text":{"text":"Foundation model AI inference served up as API endpoints will be cheap.","className":"TextPlain"},"className":"TextBold"},"className":"TextItalic"}],"className":"TextConcat"},"className":"PageBlockParagraph"},{"text":{"texts":[{"text":"PS. AI Explained on YouTube ","className":"TextPlain"},{"text":{"text":"breathlessly calls Alpaca “The Model that changes Everything","className":"TextPlain"},"url":"https://www.youtube.com/watch?v=xslW5sQOkC8","webpageId":"0","className":"TextUrl"},{"text":"” for the same reasons I think it’s important. He shares an ARK power-point slide on the lower cost of AI training and how that upends that.","className":"TextPlain"}],"className":"TextConcat"},"className":"PageBlockParagraph"},{"flags":0,"photoId":"5813331127144919018","caption":{"text":{"className":"TextEmpty"},"credit":{"className":"TextEmpty"},"className":"PageCaption"},"url":null,"webpageId":null,"className":"PageBlockPhoto"}],"photos":[{"flags":0,"hasStickers":false,"id":"5810895120478875607","accessHash":"-6116361152500356769","fileReference":{"type":"Buffer","data":[0,100,64,217,73,106,16,76,88,140,145,147,27,237,254,71,95,199,113,224,228]},"date":1679041262,"sizes":[{"type":"i","bytes":{"type":"Buffer","data":[1,27,40,202,2,158,22,133,21,42,175,173,80,13,9,79,194,180,96,142,189,234,80,153,21,33,140,5,0,118,20,175,173,135,109,10,155,113,69,76,233,138,42,132,64,130,164,43,134,86,253,42,52,233,83,198,160,182,15,76,122,210,181,208,33,80,115,184,16,70,113,86,64,1,121,168,226,141,86,65,129,138,147,2,170,48,73,141,178,57,0,162,149,197,21,77,8]},"className":"PhotoStrippedSize"},{"type":"m","w":320,"h":213,"size":7913,"className":"PhotoSize"},{"type":"x","w":794,"h":529,"sizes":[4389,6321,7722,10456,24141],"className":"PhotoSizeProgressive"}],"videoSizes":null,"dcId":4,"className":"Photo"},{"flags":0,"hasStickers":false,"id":"5810980354104864747","accessHash":"7287722309757204561","fileReference":{"type":"Buffer","data":[0,100,64,217,73,253,233,53,157,111,178,175,158,149,133,85,127,45,23,191,34]},"date":1679041262,"sizes":[{"type":"i","bytes":{"type":"Buffer","data":[1,28,40,215,44,115,129,70,88,14,104,60,49,60,82,19,184,99,35,154,0,92,177,233,210,134,108,117,160,146,40,234,115,198,104,0,82,79,210,138,3,100,227,138,40,1,10,238,99,184,241,74,64,10,112,40,52,99,29,232,1,2,1,215,147,65,1,155,158,148,1,158,244,184,227,173,0,0,225,176,7,20,82,14,163,222,138,0]},"className":"PhotoStrippedSize"},{"type":"m","w":320,"h":222,"size":23504,"className":"PhotoSize"},{"type":"x","w":800,"h":554,"size":98144,"className":"PhotoSize"},{"type":"y","w":977,"h":676,"sizes":[7918,26232,65260,88896,127444],"className":"PhotoSizeProgressive"}],"videoSizes":null,"dcId":4,"className":"Photo"},{"flags":0,"hasStickers":false,"id":"5811060111647551395","accessHash":"-7749841458283727779","fileReference":{"type":"Buffer","data":[0,100,64,217,73,83,230,202,67,58,255,37,36,93,123,210,45,183,129,12,220]},"date":1679041262,"sizes":[{"type":"i","bytes":{"type":"Buffer","data":[1,16,40,213,39,15,212,253,41,251,133,52,12,251,126,20,226,112,185,160,6,174,55,28,103,154,112,206,122,83,65,207,241,103,240,167,208,4,88,147,205,206,126,79,74,42,90,40,21,143]},"className":"PhotoStrippedSize"},{"type":"m","w":320,"h":133,"size":9255,"className":"PhotoSize"},{"type":"x","w":800,"h":333,"size":38732,"className":"PhotoSize"},{"type":"y","w":1280,"h":533,"size":78575,"className":"PhotoSize"},{"type":"w","w":1456,"h":606,"sizes":[6741,17805,47022,63826,90179],"className":"PhotoSizeProgressive"}],"videoSizes":null,"dcId":4,"className":"Photo"},{"flags":0,"hasStickers":false,"id":"5813331127144919018","accessHash":"-6481948650591900879","fileReference":{"type":"Buffer","data":[0,100,64,217,73,211,195,93,60,24,187,60,251,148,148,37,59,126,197,233,83]},"date":1679041270,"sizes":[{"type":"i","bytes":{"type":"Buffer","data":[1,22,40,213,218,123,49,253,41,118,182,62,249,253,63,194,163,185,97,246,105,113,199,20,90,31,244,104,249,231,20,237,165,192,144,18,7,57,63,133,59,39,210,141,212,185,20,128,74,41,114,40,160,12,233,37,14,140,135,60,210,36,194,56,130,12,224,14,180,81,79,160,10,147,225,112,119,31,115,74,46,49,234,104,162,134,2,253,163,158,244,81,69,32,63]},"className":"PhotoStrippedSize"},{"type":"m","w":320,"h":178,"size":12344,"className":"PhotoSize"},{"type":"x","w":800,"h":446,"size":49251,"className":"PhotoSize"},{"type":"y","w":1153,"h":643,"sizes":[6899,16148,34923,51976,79181],"className":"PhotoSizeProgressive"}],"videoSizes":null,"dcId":4,"className":"Photo"}],"documents":[],"views":null,"className":"Page"},"attributes":null,"className":"WebPage"},"className":"MessageMediaWebPage"},"replyMarkup":null,"entities":[{"offset":0,"length":70,"className":"MessageEntityUrl"}],"views":261,"forwards":0,"replies":{"flags":0,"comments":false,"replies":0,"repliesPts":234561,"recentRepliers":null,"channelId":null,"maxId":null,"readMaxId":null,"className":"MessageReplies"},"editDate":null,"postAuthor":null,"groupedId":null,"reactions":null,"restrictionReason":null,"ttlPeriod":null,"className":"Message"}

Curve DAO Token telegram news 17 March 2023 08:26