cant enable streaming + settings reverting + response issues

[obi-o-n-e]

Hi

I cant seem to enable enable streaming,
If i set it in the "stream_responses: true" in the .yaml or within the web ui - Settings will revert to defaults as soon as i press start bot within the webui.

My wake word that I custom made - seem to reset every time i refresh the webui (even if they are set in .yaml before launching ooba)

More than half of the responses dont come trough:

Is there a filter thats being triggered ? I cant seem to find any documentation 🙏

Running the Latest build available. (both ooba and bot)
TheBloke_Chronos-Hermes-13B-SuperHOT-8K-GPTQ

Funny thing is that it was working perfectly on friday, i dont know what changed overnight

[chrisrude]

Sorry to hear that!

First a caveat -- .yaml editing can be a little weird if you're trying to change settings through both the web UI and editing the config file on disk. The reason is that it's actually loading the settings from the config file into the UI widgets, and then using those values to re-write the config file each time you "Start" the bot. So any changes you made to the config file itself, once the UI is running, will be overwritten.

What should work is either:

• setting the values in the UI itself
• using the "Advanced" tab to edit the yaml
• editing the yaml while the oobabot plugin isn't running at all -- either because the Obabooga service isn't running, or because the oobabot-plugin itself is disabled.

That said, it sounds like things still aren't working, which is weird. A few questions:

• has anything changed since Friday?
• what happens if you restart the Oobabooga service entirely?

[chrisrude]

Also, could you please attach the contents of the oobabot-config.yml file? You'll want to manually remove your discord token before doing so.

[chrisrude]

Regarding the Streaming setting specifically, I'm able to reproduce that, so I should be able to get a fix in.

So what I'm most curious about now is the keywords not being saved.

[obi-o-n-e]

I have since reinstalled (before your commit though)
Made a backup of the yaml just in case, so see attached

[obi-o-n-e]

Cant figure out how to attach file (still new to github kinda)

# Welcome to Oobabot!
#
# This is the configuration file for Oobabot.  It is a YAML file, and
# comments are allowed.  Oobabot attempts to load a file named
# "config.yml" from the current directory when it is run.
#

version: 0.1.9

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# persona
# .
persona:

  # Name the AI will use to refer to itself
  #   default: oobabot
  ai_name:

  # This prefix will be added in front of every user-supplied request.  This is useful for
  # setting up a 'character' for the bot to play.  Alternatively, this can be set with the
  # OOBABOT_PERSONA environment variable.
  persona:

  # Path to a file containing a persona.  This can be just a single string, a json file in
  # the common "tavern" formats, or a yaml file in the Oobabooga format.  With a single
  # string, the persona will be set to that string.  Otherwise, the ai_name and persona will
  # be overwritten with the values in the file.  Also, the wakewords will be extended to
  # include the character's own name.
  #   default:
  persona_file: P:\ai\oobabooga_windows\text-generation-webui\characters\TavernAI-Tataru
    Taru.json

  # One or more words that the bot will listen for. The bot will listen in all discord
  # channels it can access for one of these words to be mentioned, then reply to any
  # messages it sees with a matching word. The bot will always reply to @-mentions and
  # direct messages, even if no wakewords are supplied.
  #   default: ['oobabot']
  wakewords:
    - Tataru
    - Taru

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# discord
# .
discord:

  # Token to log into Discord with.  For security purposes it's strongly recommended that
  # you set this via the DISCORD_TOKEN environment variable instead, if possible.
  discord_token: 

  # Post the entire response as a single message, rather than splitting it into separate
  # messages by sentence.
  #   default: False
  dont_split_responses: true

  # Number of lines of chat history the AI will see when generating a response.
  #   default: 7
  history_lines: '6'

  # If set, the bot will not respond to direct messages.
  #   default: False
  ignore_dms:

  # Set the log level.  Valid values are:
  # CRITICAL, ERROR, WARNING, INFO, DEBUG
  #   default: DEBUG
  log_level:

  # If set, the bot will generate a thread to respond in if it is not already in one.
  #   default: False
  reply_in_thread:

  # A list of strings that will cause the bot to stop generating a response when
  # encountered.
  #   default: ['### End of Transcript ###<|endoftext|>', '<|endoftext|>']
  stop_markers:

  # FEATURE PREVIEW: Stream responses into a single message as they are generated. Note: may
  # be janky
  #   default: False
  stream_responses:

  # FEATURE PREVIEW: Adds a limit to the number of channels the bot will post unsolicited
  # messages in at the same time.  This is to prevent the bot from being too noisy in large
  # servers.  When set, only the most recent N channels the bot has been summoned in will
  # have a chance of receiving an unsolicited message.  The bot will still respond to
  # @-mentions and wake words in any channel it can access.  Set to 0 to disable this
  # feature.
  #   default: 3
  unsolicited_channel_cap:

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# oobabooga
# .
oobabooga:

  # Base URL for the oobabooga instance.  This should be ws://hostname[:port] for plain
  # websocket connections, or wss://hostname[:port] for websocket connections over TLS.
  #   default: ws://localhost:5005
  base_url: ws://localhost:5005/

  # Print all AI input and output to STDOUT.
  #   default: False
  log_all_the_things:

  # A regex that will be used to extract message lines from the AI's output.  The first
  # capture group will be used as the message.  If this is not set, the entire output will
  # be used as the message.
  #   default:
  message_regex:

  # A dictionary which will be passed straight through to Oobabooga on every request.  Feel
  # free to add additional simple parameters here as Oobabooga's API evolves. See
  # Oobabooga's documentation for what these parameters mean.
  request_params:
    max_new_tokens: 300
    do_sample: true
    temperature: 0.7
    top_p: 0.9
    typical_p: 1
    repetition_penalty: 1.15
    top_k: 20
    min_length: 0
    no_repeat_ngram_size: 0
    num_beams: 1
    penalty_alpha: 0
    length_penalty: 1
    early_stopping: false
    seed: -1
    add_bos_token: true
    truncation_length: 1000
    ban_eos_token: false
    skip_special_tokens: true
    stopping_strings: []

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# stable_diffusion
# .
stable_diffusion:

  # When one of these words is used in a message, the bot will generate an image.
  #   default: ['draw me', 'drawing', 'photo', 'pic', 'picture', 'image', 'sketch']
  image_words:

  # URL for an AUTOMATIC1111 Stable Diffusion server.
  #   default:
  stable_diffusion_url: http://127.0.0.1:7860/

  # This will be appended to every image generation prompt sent to Stable Diffusion.
  #   default:
  extra_prompt_text:

  # A dictionary which will be passed straight through to Stable Diffusion on every request.
  # Feel free to add additional simple parameters here as Stable Diffusion's API evolves.
  # See Stable Diffusion's documentation for what these parameters mean.
  request_params:
    do_not_save_samples: true
    do_not_save_grid: true
    negative_prompt: animal harm, suicide, loli, nsfw
    negative_prompt_nsfw: animal harm, suicide, loli
    steps: 30
    width: 512
    height: 768
    sampler: DDIM

  # FEATURE PREVIEW: If set, the bot will ask Oobabooga to generate image keywords from a
  # user's message.  It will then pass the keywords that Oobabooga produces to Stable
  # Diffusion to finally generate an image. Otherwise, the bot will simply extract keywords
  # directly from the user's message using a simple regex.
  #   default: False
  use_ai_generated_keywords:

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# template
# .
template:

  # The main prompt sent to Oobabooga to generate a response from the bot AI.  The AI's
  # reply to this prompt will be sent to discord as the bot's response.
  # .
  # Allowed tokens: {AI_NAME}, {IMAGE_COMING}, {MESSAGE_HISTORY}, {PERSONA}
  # .
  #   default:  You are in a chat room with multiple participants. Below is a transcript of
  # recent messages in the conversation. Write the next one to three messages that you would
  # send in this conversation, from the point of view of the participant named {AI_NAME}.
  # {PERSONA}  All responses you write must be from the point of view of {AI_NAME}. ###
  # Transcript: {MESSAGE_HISTORY} {IMAGE_COMING}
  prompt:

  # Part of the AI response-generation prompt, this is used to render a single line of chat
  # history.  A list of these, one for each past chat message, will become {MESSAGE_HISTORY}
  # and inserted into the main prompt
  # .
  # Allowed tokens: {USER_MESSAGE}, {USER_NAME}
  # .
  #   default:  {USER_NAME} says: {USER_MESSAGE}
  prompt_history_line:

  # Part of the AI response-generation prompt, this is used to inform the AI that it is in
  # the process of generating an image.
  # .
  # Allowed tokens: {AI_NAME}
  # .
  #   default:  {AI_NAME}: is currently generating an image, as requested.
  prompt_image_coming:

  # Sent to Oobabooga, along with the user's image request, to generate             image
  # keywords.  The AI's response to this prompt will then be sent             to Stable
  # Diffusion to generate an image.
  # .
  # Allowed tokens: {AI_NAME}, {USER_MESSAGE}
  # .
  #   default:  Below is an instruction that describes a task, paired with an input that
  # provides further context. Write a response that appropriately completes the request.
  # ### Instruction: The input text asks for an image to be created.  Please create a list
  # of keywords to describe that image.  ### Input: {USER_MESSAGE}  ### Response:
  prompt_image_keywords:

[obi-o-n-e]

just an fyi , i only edit the .yaml file before launching ooba . By changes reverting is usually happens within the webui.
I.e:
stop the bot , change wake words, mash save , start bot , then would notice the wake words didnt change when i talk to bot in my server.
But if i stop it , update & save again eventually it would save.

Also notices on the initial load up of a persona card on refresh of the webui it would overwrite the wake words (rinse repeat)

Hope it shed some light

[harlyh]

I have exactly the same issue as OP described when he opened this thread.

I finally gave up and now running the bot in stand-alone mode.

[chrisrude]

Thanks for sharing the config file! I suspect it's related to having a Character selected. Will look into reproducing it with that in mind.

[obi-o-n-e]

To follow up on the streaming thing ,

After a restart of ooba server the streaming doesnt work again the setting will revert to blank (false)

Weird , as i havent changed anything in ooba apart from loading a different model.

As to a wake word issue its definately a Character Card override - I can work around by changing the default card name.

[chrisrude]

cool. I reproduced the issue, which appears to be with .json-formatted character cards. I hadn't been testing with that, and it was quite broken!

The mess was with the logic to automatically add the character's name as a wakeword, which was dropping user-supplied wakewords in some cases. I decided to just make this all simpler and let the wakewords be wakewords, and trust that the bot user can figure out themselves what they want to enter there.

This should be fixed in the above commit, which should be included in the next release, which I hope to get out sometime this week.

[obi-o-n-e]

Nice nice , looking forward to seeing it.

Have you managed to reproduce the issue in regards to the streaming setting reverting after ooba gets restarted/reloaded.

Cause until i uninstall the plugin and reinstall - Streaming only works for the 1st launch of the ooba. If it gets restarted , then it defaults back to false.

But anyway , i can live without it. Have had great deal of use from the bot as it is already. 👍