I’ve been tinkering on an idea with some colleagues regarding “LLMs as first class citizens to the OS” – when this paper was released recently https://arxiv.org/pdf/2310.08560.pdf -“MEMGPT: TOWARDS LLMS AS OPERATING SYSTEMS”, the title sounded exciting, but the paper is a bit different than I anticipated.

Regardless I set the project up https://github.com/cpacker/MemGPT and started hacking around.

Getting started was pretty easy:

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt
export OPENAI_API_KEY=<YOUR_API_KEY>

python3 main.py

The default setup is pretty neat – I had a surreal conversation w/the llm and the length of memory was impressive. MemGPT works by keeping some context in memory, and also moving data to a vectorDB. (Read the paper for the details)

In the example, they show the ability to take in a .db (assuming sqlite here…); however in practice this doesn’t work.

Here’s a simple way to get the .csv into .db format:

sqlite> PRAGMA encoding;
encoding
--------
UTF-8
sqlite> .mode csv
sqlite> .separator ","
sqlite> .import somedata.csv tbl
sqlite> .schema tbl
CREATE TABLE IF NOT EXISTS "tbl"(
"salesdate" TEXT, "customer" TEXT, "customername" TEXT, "customercountry" TEXT,
 "customerregion" TEXT, "customercity" TEXT, "customerlocation" TEXT, "corporatecustomercode" TEXT,
 "cccname" TEXT, "globalindicatorgroup" TEXT, "marketsegment" TEXT, "industry" TEXT,
 "industryname" TEXT, "customerclassification" TEXT, "micc" TEXT, "miccname" TEXT,
 "material" TEXT, "division" TEXT, "salesorganization" TEXT, "salesoffice" TEXT,
 "profitcenter" TEXT, "vendor" TEXT, "transactiontype" TEXT, "currency" TEXT,
 "revenue" TEXT, "landedgrossmargin" TEXT, "unit" TEXT, "quantity" TEXT);
sqlite> .save out.db
sqlite> .quit

We confirm we are using UTF-8 and save the db from memory locally.

Now lets attempt to bring that into MemGPT:

python main.py --archival_storage_files=out.db
...bunch of errors...
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 31: invalid continuation byte

So I spent a while messing around to fix this before digging into the MemGPT code. After digging into we see that it’s just trying to read the .db file directly…so might as well just serve it as a .csv

def chunk_file(file, tkns_per_chunk=300, model='gpt-4'):
    encoding = tiktoken.encoding_for_model(model)
    with open(file, 'r') as f:
        if file.endswith('.pdf'):
            lines = [l for l in read_pdf_in_chunks(file, tkns_per_chunk*8)]
            if len(lines) == 0:
                print(f"Warning: {file} did not have any extractable text.")
        elif file.endswith('.csv'):
            lines = [l for l in read_in_rows_csv(f, tkns_per_chunk*8)]
        else:
            lines = [l for l in read_in_chunks(f, tkns_per_chunk*4)] <-- execution

def read_in_chunks(file_object, chunk_size):
    while True:
        data = file_object.read(chunk_size) <-- fails
        if not data:
            break
        yield data

So lets go back and try something even easier!

main.py --archival_storage_files=somedata.csv
⚙️ Using legacy command line arguments.
None gpt-4 ('sam_pov', 'memgpt/personas/examples')
Running... [exit by typing '/exit']
Preloaded 13625 chunks into archival memory.
Initializing InMemoryStateManager with agent object
InMemoryStateManager.all_messages.len = 4
InMemoryStateManager.messages.len = 4
💭 Bootup sequence complete. Persona activated. Testing messaging functionality.
Hit enter to begin (will request first MemGPT message)

Well this sort of works, back to that later. Lets actually go back and fix the code to support the sqlite db.

Basically the fix should be:

    if file.endswith('.db'):
        lines = read_database_as_list(file)
    else:
        with open(file, 'r') as f:
            if file.endswith('.pdf'):
                lines = [l for l in read_pdf_in_chunks(file, tkns_per_chunk*8)]
                if len(lines) == 0:
                    print(f"Warning: {file} did not have any extractable text.")
            elif file.endswith('.csv'):
                lines = [l for l in read_in_rows_csv(f, tkns_per_chunk*8)]
            else:
                lines = [l for l in read_in_chunks(f, tkns_per_chunk*4)]

So I kicked this over to the MemGPT maintainers: https://github.com/cpacker/MemGPT/pull/106

Thanks, and until next time! sk out!

(Shout to Kyle H, wirepair & galt from #innercircle for their help and motivation)