TypeError in Python 3.11 when Using BasicModelRunner from llama-cpp-python

Question

I'm currently taking the DeepAI's Finetuning Coursera course and encountered a bug while trying to run one of their demonstrations locally in a Jupyter notebook.

Environment:

Python version: 3.11
Required packages:
- notebook
- lamini
- llama-cpp-python (> 0.1.53)

Issue:

When attempting to run the following code:

import os
import lamini

lamini.api_url = os.getenv("POWERML__PRODUCTION__URL")
lamini.api_key = os.getenv("POWERML__PRODUCTION__KEY")

from llama import BasicModelRunner

non_finetuned = BasicModelRunner("meta-llama/Llama-2-7b-hf")

non_finetuned_output = non_finetuned("Tell me how to train my dog to sit")

I receive a TypeError at the last line:

TypeError: can only concatenate str (not "NoneType") to str

This error occurs when I try to pass a string to the non_finetuned BasicModelRunner object.

Friedrich · Accepted Answer · 2024-05-17 08:12:21Z

1

I had the same problem. You have to create an account in lamini then you can create a key copy/past to POWERML__PRODUCTION__KEY for POWERML__PRODUCTION__URL let it as empty string ""

edited May 17, 2024 at 8:12

Friedrich

5,45716 gold badges82 silver badges62 bronze badges

answered May 16, 2024 at 13:10

walid khallef

262 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Hofbr · Accepted Answer · 2024-01-27 21:52:41Z

0

The environment variables POWERML__PRODUCTION__URL and POWERML__PRODUCTION__KEY were not properly set.

answered Jan 27, 2024 at 21:52

Hofbr

1,0201 gold badge18 silver badges43 bronze badges

Collectives™ on Stack Overflow

TypeError in Python 3.11 when Using BasicModelRunner from llama-cpp-python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related