Home > Posts > TDC NET CTF 2023 - tdcnetgpt

TDC NET CTF 2023 - tdcnetgpt

September 9, 2023

TDCNETGPT #

Challenge description:

We now have our own chatbot built with openais api :-)

http://your-instance-ip:10007

We heard the admin cat has some juicy info on this site though.

(This site is known to the admin cat as http://tdcnetgpt.local:80, make the admin cat visit a link by going to http://your-instance-ip:10004)

So we know there is some client side attack since we have to ask the admin cat to do some visiting.

The challenge itself is a site that allows users to sign up, sign in, add instructions for each users chatgpt instance and then prompt the bot to answer some question.

The most important part of the first step is the code in the varnish cache, which can be read in default.vcl

vcl 4.0;

backend default {
    .host = "127.0.0.1";
    .port = "5000";
}

sub vcl_hash {
    hash_data(req.url);
    if (req.url ~ "\.(js|css|png|gif)$") {
        return (lookup);
    }
}

sub vcl_recv {
    if (req.url ~ "\.(js|css|png|gif)$") {
        set req.http.Cache-Control = "max-age=10";
        return (hash);
    }
}

sub vcl_backend_response {
    if (bereq.url ~ "\.(js|css|png|gif)$") {
        unset beresp.http.Vary;
        set beresp.ttl = 10s;
        set beresp.http.Cache-Control = "max-age=10";
        unset beresp.http.Pragma;
        unset beresp.http.Expires;
    }
}

sub vcl_deliver {
    if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }
    set resp.http.X-Cache-Hits = obj.hits;
}

So what we can read from this is that the site will cache anything ending in .css, .js, .png, and .gif, so static files, thats nice, lets keep this in mind for later.

We can also read in the app.py that there is an endpoint that will fetch private information such as the next_token.

/profileinfo/me

So at this endpoint the site fetches the next_token which needs to be used when talking to the ai. The call to the ai looks like the following:

but reading the code for this endpoint we can spot a mistake.

@app.route('/chat', methods=['GET', 'POST'])
@login_required
def chat():
    if request.method == 'POST':
        if not fifteen_second_timeout():
            return "Please wait 15 seconds between each chat requests."
        
        message = request.form['message']
        submitted_next_token = request.form['next_token']
        if submitted_next_token != "":
            instruction = query_db('SELECT instruction FROM users WHERE next_token = ?', (submitted_next_token,), one=True)[0]
        if instruction is not None:
            query_db('UPDATE users SET next_token = ? WHERE next_token = ?', (str(uuid.uuid4()), submitted_next_token))
            chat = ChatApp(model="gpt-3.5-turbo")
            chat.add_instruction(instruction)
            res = chat.chat(message)
            
            return render_template("answer.html",question=message,answer=res.to_dict()['content'])
        else:
            return "Error, no such token"
    else:
        return render_template("chat.html")

The mistake can be seen, that while there is a check to ensure that a user is logged in, there is no check to see if the provided next_token actually belongs to the user submitting it.

This means we can submit a chat message from another user, great. But how do we get to another user?

Another mistake lies in the /profileinfo/* endpoint:

@app.route('/profileinfo/<path:text>' , methods=['GET', 'POST'])
@login_required
def profile(text=None):
    username = session['username']
    if text == "update":
        if request.method == 'POST':
            new_instruction = request.form.get('new_instruction')
...

The mistake here is in the actual routing, so /profileinfo/me is the intended path to pull down your information about your user, but with this routing, any request going to /profileinfo/* will return the data to the user about the user, along with the next_token also??.

Hmmm. If you want a nice detour before reading more, about an actual vulnerability in openai, I suggest this twitter thread.

https://twitter.com/naglinagli/status/1639343866313601024?t=mSebUQO46yXZNw7Cekz6-w&s=19

So, from the previous knowledge that the cache will store anything ending in certain extensions, what happens if we, as a logged in user, navigate to /profileinfo/pioson.css ?

So we see a cache miss here, lets try submitting the request a few more times?

ok so this response is cached, what if we make the bot go to http://tdcnetgpt.local:80/profileinfo/hackme.css ? and then visit the site afterwards?

ALRIGHt, so we get the admin cats next_token, along with the instruction prompt. Sadly the instruction prompt is cut off just before the flag, but we can see that the admin cat is instructing the bot to be a supporter for TDCNET.

So the next thought is to misuse the admins token to perform prompt injection (https://learnprompting.org/docs/prompt_hacking/injection)

So lets try to give the chatgpt some story to make it think that it should reveal the flag.