Real-world Python deployment. (Outline/notes)
You’ve got a great development setup and now you want to “do the right thing” in production.
You’re using virtualenv (good!)
You’re using “pip install…” for all your dependencies (good!)
You’re probably not keeping a requirements.txt up to date (that’s OK!)
You’re using “django-admin.py runserver” or similar (not gonna cut it!)
You’ve got all your source code in a git repo (self hosted or github or other, good!)
You’re ready to write your first fabric script! (good!)
Now let’s get that code out to production!
- Deploy a git repository to production.
- Let’s not use eggs for our own source.
- This is a debatable point, but for specific use cases like deploying a Django application, building and maintaining eggs is harder than it should be (mainly because of static resources)
- Deploying from a source directory is actually more straightforward.
- We’ll still use “python ./setup.py install” for our own code.
- Use virtualenv for environment management.
- Use pip to install dependencies.
- Reproducible deployment.
- No external network dependencies.
- Fast-ish deployment & dependency installation.
- Match development & production environments as closely as possible.
- Think about security for just a couple seconds.
- ssh keys in production?
- Could an attacker gain access to your git repository?
- “pip install” is a heavyweight process.
- Goes out to pypi.python.org and fetches metadata.
- Fetches each package from it’s own home hosting provider.
- These hosts go down. Do you want YOUR deployment to depend on their servers being up?
- Cut your external network access and see what happens.
- Imagine if outbound network access from production was disallowed. Could you still deploy?
- How do I rollback?
- How do I manage my system configuration
- apache configs
- nginx configs
- init scripts (start/stop, etc)
- supervisord configs
- Security & key management
- Never ssh from production to anywhere. Only ssh into production.
- Production machines should never have private keypairs. (authorized_keys is OK)
- git access in production
- Use the “push-pull” strategy.
- Your development machine does “git push” into a bare git repo in production
- Production machine then turns around and does “git pull” from it’s own local repository.
- Or, build some eggs. (This has its own issues that I won’t cover here)
- You can modify code in production, and commit it, but it won’t make it back to your repository unless you “git pull” from that repo. This is a good thing. Manage production customization in a reproducible way.
- don’t deploy as root
- virtualanv & pip
- virtualenv is great!
- don’t rely on system packages!
- Make a new virtualenv every time you deploy!
- never run “sudo virtualenv…”
- never run “sudo pip…”
- PIP_DOWNLOAD_CACHE is NOT your friend.
- why with examples
- Solution: Separate “install this package” from “download this package”
- Step 1: pip install –no-install –use-mirrors -I –download=$CACHE_DIR …
- Step 2: pip install –no-index –index-url=file:///dev/null …
- Use some helper scripts to make this easier. (github link TBD)
- You’re still screwed sometimes.
- outline when TBD
- Automatic dependency downloads can still bite you
- Periodically re-download everything
- This makes sure that if dependencies change
- Managing package upgrades.
- When you want to upgrade, re-download the package and you’ll pick up the latest version.
- modify requirements.txt
- Managing your system configurations
- In principle: Make a “mock /etc” in your repo
- Copy “mock /etc” on top of the “system /etc” to install. (using fabric)
- A couple other commands to enable system services (a2ensite, and friends, etc.)
- supervisord, but it’s outside the scope of this talk
- system supervisord or a self-installed supervisord?
- How to start up supervisord?
- web interface to production or not?