All in all pretty decent sorry I attached a 35 min video but didn't wanna link to twitter and wanted to comment on this...pretty cool tho not a huge fan of mark but I prefer this over what the rest are doing...
The open source AI model that you can fine-tune, distill and deploy anywhere. It is available in 8B, 70B and 405B versions.
The issue is that "open source" is a term for computer software. And it doesn't really apply to other things. But people use it regardless. With software, it means you share the recipe, the program code. With machine learning models, there isn't really such a thing. It's a pile of numbers (the weights) that are the important thing. They get shared in this case. But you can't reproduce them. For that you'd need the dataset that went in (which Meta doesn't share because lots of that is copyrighted and they have several court cases running because they just stole the texts and said it's alright.) But what open source allows (amongst other things) is to build upon things and modify them. And that can be done with the models to a certain degree. They can be fine-tuned and incorporated in custom projects. In the end they (Meta) want to frame things a certain way and be the good guys. But the term still doesn't really mean what it's supposed to mean.
There are other models with other licenses. There are Apache-licensed models available. There are models which do or don't allow for commercial usage. We also have some with the datasets and everything available. But at least those aren't state of the art anymore.
iv. Your use of the Llama Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Llama Materials (available at https://ai.meta.com/llama/use-policy), which is hereby incorporated by reference into this Agreement.
v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).
Additional Commercial Terms. If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
Yeah more or less open source to these guys is just like saying they didn't close out any parts of the code...which they didn't but beyond that I agree with you totally.
Parameter count. 8 billion ... Colloquially the model size, and hence how smart it is. 405 billion parameters is big. We didn't have anything even close to that size and with current technology to download and tinker around, until just now.
I mean, from what I can tell we still don't, at least as home users. The full size model won't fit on any commercial hardware. Even with a top of the line 4090 GPU you're limited to the 8B model if you want to run it offline, and that still charts lower than the last-gen 70B model.
From the benchmarks it seems like it's actually a noticable improvement over Llama 3. Llama 3 was already a lot better than Llama 2 (from actually using it, not just benchmarks), so I'm really interested in how good this actually is in practice.
Haha well at least under the hood he seems semi normal I remember videos of him past few years with a stern look on his face talking like a robot were so cringe.... But ya
Never mind that, what the hell is up with his face? Why does he look like a negative image of someone who hasn't slept in months? Did he get replaced by his ginger cousin? Is he being played by Jesse Eisenberg again, just with a deepfake filter?