@info-creaticode You’re welcome, but I’ve considered these issues and tested them.
For one, I am using a school chromebook, and they don’t really differ much across schools except for rules, touchscreen, and sometimes specs like storage. This was when I booted up the AI:
95611ed9-ad0a-43e5-928a-7273598a1702-image.png
It is suprisingly memory efficient because of the fact that it runs on the GPU. They run just fine and great on 8GB of RAM, which is what most enterprise/school chromebooks have.
Also, I’m gonna say this, no offense, but if a chromebook can run CreatiCode, then it can absolutely run WebLLM. Memory usage with WebLLM tabs rarely go over a gigabyte for me, but that used to be pretty common with CreatiCode.
Second, I’ve tried to get these models to generate inappropriate content as a test and they won’t. That’s because these models are open-source ones from big companies like Meta and Microsoft (with Phi and Llama) on Huggingface, simply ported to the format that WebLLM supports. Even with prompts that enable it to do whatever, it chooses to stick strongly to its guidelines. Here’s an example, where it still did what I asked but not exactly:
5c80dfd0-33b6-4426-b74b-e73871986594-image.png
The “adult themes” it was able to generate was a pretty cliche one of drinking beer in a bar.
It would be hard to get this to generate inappropriate content because of that and a bit of its incompetence at some things, likely because I’m using a less powerful model (Llama-3.2-1B-Instruct-q4f16_1-MLC) for the sake of my computer and for faster load times.
So while both those points make sense, WebLLM and the built in model safeguards already handle them and if the device can’t support it, then using the normal ChatGPT API as a fallback works too. I think WebLLM is safer security/privacy-wise, like how it is all client side and running in the browser, so you don’t have to deal with data analytics on OpenAI’s backend or stuff like that. There’s also the fact that the models don’t change with WebLLM, so you won’t have to worry about some update ruining stuff (like how people have been mad about GPT-5 being a failure and many switching to Anthropic/Claude).
