The trouble with all these schemes is that it’s totally contrary to poweful real world trends. The surface of the Earth has an overwhelming abundance of rural land that is incredibly hospitable to life. And these places are depopulating because people prefer living in cities. How are you gonna get people to move to the bottom of the sea, or Mars, if they don’t even want to move to West Virginia?
LLMs aren’t capable of maintaining an even remotely convincing simulacrum of human connection,
Eh, maybe, maybe not. 99% of the human-written stuff in IM chats, or posted to social media, is superficial fluff that a fine-tuned LLM should have no problem imitating. It’s still relatively easy to recognize AI models outputs in their default settings, because of their characteristic earnest/helpful tone and writing style, but that’s quite easily adjustable.
One example worth considering: people are already using fine tuned LLMs to copilot tabletop RPGs, with decent success. In that setting, you don’t need fine literature, just a “good enough” quality of prose. And that is already far exceeding the average quality that you see in social media.
Hot take: if they can get it to work, good! I welcome AI users who are smarter, better informed, and have better taste than the rest of us mouth breathing meatbags.
It would be foolish of them to lie considering their model is open source and uploaded to a public repository. The hardware specs for running it are pretty steep, but third parties are already doing it.
The moat is probably mostly inertia. Microsoft or whoever will offer a code assistant that directs to OpenAI’s model, and users will just use that. Most software moats are like that, rather than being based on intrinsic technological superiority.
Kudos to Deepseek for continuing to releasing the code and model under a permissive license. Would be nicer if the weights were under an MIT license rather than a custom license, but I guess they’re afraid of liability. Strange situation we’re now in, where the future of open AI (as opposed to “open but actually closed” AI) now almost entirely depends on Chinese companies.
In practice, though, I wonder how many people would actually self host and tinker with this, since the model is way too large to run on any desktop. It would be very interesting to find downstream use-cases and modifications, which is supposed to be a strength of the open source model. Deepseek themselves don’t seem to be much concerned about applications; from my understanding, they are basically funded by a sugar daddy and are happy to just do R&D (funnily enough, that is kinda what OpenAI was originally supposed to be before they sold out to Microsoft).
Deepseek trained their v3 model for $6M. That’s the AI equivalent of building it in a cave with a pile of scraps. There’s no longer any reasonable way to stop China from developing frontier models.