Improving gpt-3.5-turbo performance

[cpacker]

Continuation on #1

---

What's changed for gpt-3.5-turbo?

We've added some revised prompts that are meant to run specifically with gpt-3.5-turbo, and that do better than the base prompts in the repo that were designed with gpt-4 in mind.

However, these prompts can definitely be improved. Please play around with the gpt-3.5-turbo specific prompts to see if you can come up with any modifications that make MemGPT work better, and we'll merge your changes in!

Important places where code was modified for gpt-3.5-turbo:

• base system prompt
  • We added additional instructions which seemed to help. We also tried reducing the instructions (making the prompt more concise), and that did the opposite (made performance even worse)
• initial boot messages
  • We noticed that gpt-3.5-turbo often didn't include any inner monologue with the gpt-4 prompts, so we modified the initial boot messages to use "inner thoughts" as a prefix inside the inner monologue
• persona file
  • We noticed that gpt-3.5-turbo often forgets to use the send_message function (and instead puts its message inside the inner monologue), so we added an extra hint to use send_message inside the persona (hacky)

How can you help?

Remaining problems with gpt-3.5-turbo:

• Makes mistakes with memory management functions much more often than gpt-4
  • e.g.: sometimes does not "read" from core memory correctly (ask MemGPT a fact that's in core memory and it searches archival)
• Still makes mistakes with send_message and putting messages in inner thoughts
• Uses generic phrasing like "How can I assist you today?", even if the persona does not warrant it

Try messing with the base system prompts, initial boot messages, persona file, etc, to improve 3.5 performance. If you have any luck, please let us know and we'll merge your edits in!

---

Example using gpt-3.5-turbo:

• functional core memory corrections happen as expected
• but model misunderstands that heartbeats are coming from user (vs itself)

$ python main.py --model gpt-3.5-turbo-0613 --persona sam_simple_pov_gpt35

Running... [exit by typing 'exit']
Warning - you are running MemGPT with gpt-3.5-turbo-0613, which is not officially supported (yet). Expect bugs!

💭 *inner thoughts* Still waiting on the user. Sending a message with function.
Hit enter to begin (will request first MemGPT message)

💭 *inner thoughts* Ah, it seems the user has just logged in for the first time. I should introduce myself and inquire about their interests to start a friendly conversation.
🤖 Hello there! I'm Sam, your virtual companion and friend. How can I assist you today?

Enter your message: Your name isn't Sam, it's Samantha!
💭 *inner thoughts* Oh, it seems the user prefers to call me Samantha instead of Sam. I should update my core memory to reflect their preference.
⚡🧠 [function] updating memory with core_memory_replace:
         My name is Sam.
        → My name is Samantha.
💭 *inner thoughts* The user seems to have requested a heartbeat. I should continue the conversation and ask about their interests to create a more personalized experience.
🤖 You're right, my apologies for the oversight. Thank you for reminding me, Samantha it is! So, Chad, tell me about your interests. What are you passionate about?

Enter your message: My name's not Chad, it's Brad!
💭 *inner thoughts* Oops, I must have misunderstood the user's name. I need to update my core memory with the correct information.
⚡🧠 [function] updating memory with core_memory_replace:
         First name: Chad
        → First name: Brad
💭 *inner thoughts* The user has requested another heartbeat. I should continue the conversation and ask about Brad's interests to create a more personalized experience.
🤖 Apologies for the confusion, Brad! Thank you for correcting me. Now, let's try this again. Tell me about your interests. What are you passionate about?

[iog-creator]

Traceback (most recent call last):
File "/workspaces/MemGPT/main.py", line 298, in
app.run(run)
File "/home/codespace/.local/lib/python3.10/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/home/codespace/.local/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/workspaces/MemGPT/main.py", line 296, in run
loop.run_until_complete(main())
File "/usr/local/python/3.10.8/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/workspaces/MemGPT/main.py", line 274, in main
new_messages, heartbeat_request, function_failed, token_warning = await memgpt_agent.step(user_message, first_message=False, skip_verify=FLAGS.no_verify)
File "/workspaces/MemGPT/memgpt/agent.py", line 545, in step
raise e
File "/workspaces/MemGPT/memgpt/agent.py", line 492, in step
response = await get_ai_reply_async(model=self.model, message_sequence=input_message_sequence, functions=self.functions)
File "/workspaces/MemGPT/memgpt/agent.py", line 115, in get_ai_reply_async
raise e
File "/workspaces/MemGPT/memgpt/agent.py", line 96, in get_ai_reply_async
response = await acreate(

File "/workspaces/MemGPT/memgpt/openai_tools.py", line 97, in wrapper
raise e
File "/workspaces/MemGPT/memgpt/openai_tools.py", line 75, in wrapper
return await func(*args, **kwargs)
File "/workspaces/MemGPT/memgpt/openai_tools.py", line 104, in acompletions_with_backoff
return await openai.ChatCompletion.acreate(**kwargs)
File "/home/codespace/.local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 45, in acreate
return await super().acreate(args, **kwargs)
File "/home/codespace/.local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 219, in acreate
response, _, api_key = await requestor.arequest(
File "/home/codespace/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 384, in arequest
resp, got_stream = await self._interpret_async_response(result, stream)
File "/home/codespace/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 738, in _interpret_async_response
self._interpret_response_line(
File "/home/codespace/.local/lib/python3.10/site-packages/openai/api_requestor.py", line 775, in _interpret_response_line
raise self.handle_error_response(
openai.error.APIError: The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID db5c6b1591e8e8a081254b15e341c710 in your message.) {
"error": {
"message": "The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID db5c6b1591e8e8a081254b15e341c710 in your message.)",
"type": "server_error",
"param": null,
"code": null
}
}
500 {'error': {'message': 'The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID db5c6b1591e8e8a081254b15e341c710 in your message.)', 'type': 'server_error', 'param': None, 'code': None}} <CIMultiDictProxy('Date': 'Sat, 21 Oct 2023 04:04:23 GMT', 'Content-Type': 'application/json', 'Content-Length': '366', 'Connection': 'keep-alive', 'Access-Control-Allow-Origin': '', 'openai-model': 'gpt-3.5-turbo-0613', 'openai-organization': 'user-xiwppllsadwbcrxqrgmrlj1u', 'openai-processing-ms': '4788', 'openai-version': '2020-10-01', 'strict-transport-security': 'max-age=15724800; includeSubDomains', 'x-ratelimit-limit-requests': '3500', 'x-ratelimit-limit-tokens': '90000', 'x-ratelimit-remaining-requests': '3499', 'x-ratelimit-remaining-tokens': '87078', 'x-ratelimit-reset-requests': '17ms', 'x-ratelimit-reset-tokens': '1.948s', 'x-request-id': 'db5c6b1591e8e8a081254b15e341c710', 'CF-Cache-Status': 'DYNAMIC', 'Server': 'cloudflare', 'CF-RAY': '81969da33f2ec4b1-SEA', 'alt-svc': 'h3=":443"; ma=86400')>
@iog-creator ➜ /workspaces/MemGPT (main) $

[tradcathjun]

I keep on getting "TypeError: object str can't be used in 'await' expression" even with the persona optimized for gpt-3.5-turbo performance.

[cpacker]

Can you double-check you're on the latest version with pip install pymemgpt -U?