ChatGPT is limited and not as powerful as actual ChatGPT

JaecadeJnight

Here is my project where I’ve noticed these issues:
https://play.creaticode.com/projects/67197501874571534a3b20da

I got my progress back so it’s good now but ChatGPT is really bad at some stuff.

When I go to https://chatgpt.com , it can do things like math like a million times faster and more accurate than CreatiCode’s AI blocks. I’ve been trying to make my enhanced ChatGPT and it works, but ChatGPT tends to forget everything I taught it yet remembers when asked, but can’t use the information. I’ve been following the tutorials to enhance ChatGPT for things like web search and math and I even added my own feature so it doesn’t need to be in a fixed format, so that variable can be anywhere it just needs to say |CALC:5+5^2| or something. ChatGPT was good with the math until I asked a trickier question:

I checked the console log and it didnt even do what I told it to do despite it remembering because I taught it a second time. I then showed it that it needs to write |SQRT:(5-244)/5^20| even though it was taught that at the very beginning of the conversation when I used system request to train it very specifically to remember to use variables. Same thing happened here:

And it should know how to do what im asking since I told it to:

I want it to use what I told it to use without me having to give a constant and extremely specific reminder throughtout the conversation as the user.

I think ChatGPT may also be outdated or not using current gpt4.

Tyller_

@jeffreyrb03-gmail a few things,
First, I recommend setting temperature to 0 for training, since you can add on “respond with okay” to each training prompt so it should minimize the token count of the training (some is better than none and it should ensure it doesn’t think the user knows the prompt stuff).
Second, it does indeed rarely work, hypothetically you could select another chat bot and then ask it if the primaries one response could require one of the tools, it would be much more inefficient though.
Third, when it did do websearch it got stuck somewhere in the web check block (you can take apart running code that’s stuck and the block it’s stuck on will remain highlighted yellow, the rest stop being yellow), you need to update the regex to check for the “:”, that way it doesn’t see WEB and WEBSEARCH as the same (I dug further and found it gets stuck on the fetch page as markdown block, currently working on finding out why it takes so long to do that, it seems that it’s trying to check forums cus that’s what you put in last, but my console bookmarklet has it eventually error vm,warn,Primitive rejected promise: ,TypeError: undefined is not an object (evaluating 'e.t0.response.status'), I’d recommend checking dev console cus I have no clue what it’s doing there)
Fourth thing I noticed, I’d assume it’s using gpt3.5 or something, because the token limit for gpt4 I believe was something like 126 thousand (or somewhere that high) and you can hit the token limit way below that (at I’d assume 4 thousand), also the slow speed is most definitely api lag, it may just be normal ChatGPT doesn’t properly do tools like that, you could try to tell normal ChatGPT to use those tools and see if it actually works (Ik it wouldn’t at all be the same, since Chatgpt.com doesn’t use system prompts, but it’s worth a shot)

JaecadeJnight

@tyller_ These are some good ideas to optimize the code and make it better, but I dont think it’s gonna fix ChatGPT’s current problems/annoyingness especially when you try to explain to it what it’s doing wrong but ChatGPT is still stubborn and just ignores you and continues to do what it’s doing wrong

CreatiCode

@jeffreyrb03-gmail

The ChatGPT API you use is the same as the one running on https://chatgpt.com/, except there are additional moderations on its input and output.

You are dealing with a very complex task, so it will take some efforts to get the prompt right.

Here are some suggestions worth trying:

You have a few “train” blocks, which essentially just give ChatGPT some additional system prompts. It is strongly recommended that all of them be combined into one longer prompt instead of having a few pieces. The reason is that each time you send a system request, ChatGPT will generate a response, and you are ignoring those responses.

As explained in our tutorials, it is much better to use a text editor (like the comment box or just any text editor), write the entire prompt in there, and then copy it into the ChatGPT block.

You are trying to teach it to do quite a few things. Usually it is better to write the examples as dialogs:

user: …
you: …
user: …
you: …

This is much easier for ChatGPT to understand

Please make sure the prompts are self-explanatory. For example, this part is not very clear: “Now make sure to remember your variables”. Since these are special tools for callback, not really “variables”, right?
Since you are composing a super long prompt, it is better to have sections with headers, such as these:

<Overview>

<Tools you can callback>

<Current Task>

Start simple and try to add more. For example, make sure “CALC” is working fine, then add one more tool, then add another one. This will allow you to catch any issues early on. Currently, it doesn’t work for CALC yet:

JaecadeJnight

@info-creaticode Ohh that explains it. I’m gonna use these suggestions make the train blocks obsolete by just putting it all into one text with a text editor.
The calc not working is weird, last time I checked it did work but I may have broken it after tinkering with my code too much.

JaecadeJnight

@info-creaticode I have fixed it and it works better by following the suggestions of:

condensing it all into one prompt by removing the train blocks and using a comment to write the prompt
fixing the prompt and instead of calling them variables i called them “special commands”, as well as making it very clear by using headers and lists and examples
fixing the commands to work and get replaced ( still a working in progress but works for most simple cases)

but how would you recommend fixing this:

ChatGPT keeps on saying “If you need help or assistance let me know” which is annoying because it says that in practically every single message.

Tyller_

@jeffreyrb03-gmail Check if it does that on the normal ChatGPT website? I think I remember ChatGPT normally being very affirmative that it’s there to help you.

CreatiCode

@jeffreyrb03-gmail

What woud you like it to say instead in this situation?

JaecadeJnight

@info-creaticode Actually I managed to do that it’s fine now. Thank you for the suggestions on fixing my bot, they really helped!

CreatiCode

@jeffreyrb03-gmail

Great.

The True ODST

@jeffreyrb03-gmail Here is a screenshot from the chatgpt I use.
This site eventually stops the chat bot from talking

It also works as a challenge:

Though, once it think it “wins,” it keeps trying to play the game, which I think was dumb.