Using cookies.sqlite in wget or curl

Newer versions of FireFox use a sqlite3 database to store their cookies.

Older versions of FireFox used a .txt file.  wget and curl know how to parse the older style text file, but not the sqlite3 database.

So, I wrote a quick script to extract the cookies.sqlite database and generate a file that looks just like the old cookies.txt file, and then you can pass that to wget or curl. Here’s the script:

#!/bin/bash

function cleanup {
rm -f $TMPFILE
exit 1
}

trap cleanup  SIGHUP SIGINT SIGTERM

# This is the format of the sqlite database:
# CREATE TABLE moz_cookies (id INTEGER PRIMARY KEY, name TEXT, value TEXT, host TEXT, path TEXT,expiry INTEGER, lastAccessed INTEGER, isSecure INTEGER, isHttpOnly INTEGER);

# We have to copy cookies.sqlite, because FireFox has a lock on it 
TMPFILE=`mktemp /tmp/cookies.sqlite.XXXXXXXXXX`
cat $1 >> $TMPFILE
sqlite3 -separator '	' $TMPFILE << EOF
.mode tabs
.header off
select host,
case substr(host,1,1)='.' when 0 then 'FALSE' else 'TRUE' end,
path,
case isSecure when 0 then 'FALSE' else 'TRUE' end,
expiry,
name,
value
from moz_cookies;
EOF
cleanup

Usage should be like this:

$ extract_cookies.sh $HOME/.mozilla/firefox/*/cookies.sqlite > /tmp/cookies.txt
$ wget --load-cookies=/tmp/cookies.txt http://mysite.com
$ # OR 
$ curl --cookie /tmp/cookies.txt http://mysite.com 

You could use this script to wrap curl or wget itself in a similar extraction scheme, so that you always get the most current cookie values when you run wget.

Tagged , , , , , ,

10 thoughts on “Using cookies.sqlite in wget or curl

  1. Oyster says:

    Hey, thanks for this post! I’ve been meaning to learn some SQLite. This worked perfectly with wget.

  2. tai viinikka says:

    Yes, this is a great example of how to deal with sqlite as the cookie keeper. Thanks a lot!

  3. Ale-x says:

    cool, thank you for this script :)

  4. dttocs says:

    Thanks. One minor problem, if there’s a space in your cookie jar path, the script fails.

    To fix it, change:

    cat $1 >> $TMPFILE

    to

    cat “$1″ >> $TMPFILE

  5. Zaid Abdulla says:

    I’ve written a Firefox extension that makes this easier: https://addons.mozilla.org/en-US/firefox/addon/cliget/

  6. Hercules van Tonder says:

    Nice for UNIX users, but how about us poor Windows relations ?

  7. Elliott says:

    Hercules, you can install Cygwin and reap the power of UNIX!

  8. [...] use a convenient script to send all your current Firefox cookies from an host into curl. Tagged: [...]

  9. Richard says:

    Awesome plugin, I use it almost every day. Thank you so much, Zaid!
    I didn’t know that you have to type ‘about:config’ into the URL bar to get to the firefox configuration attributes hope this information helps others as well.

  10. pohuykto says:

    Not working!
    error message “file is encrypted or is not a database”
    firefox 20

Leave a Reply