3. Users and directories¶
Right now your Django project is at /root
, or maybe at
/home/joe
. The first thing we are going to fix is put your Django
project in a proper place.
I will be using $DJANGO_PROJECT
as the name of your Django
project.
3.1. Creating a user and group¶
It’s a good idea to not run Django as root. We will create a user
specifically for that, and we will give the user the same name as the
Django project, i.e. $DJANGO_PROJECT
. However, in principle it can
be different, and I will be using $DJANGO_USER
to denote the user
name, so that you can distinguish when I’m talking about the user and
when about the project.
Execute this command:
adduser --system --home=/var/opt/$DJANGO_PROJECT \
--no-create-home --disabled-password --group \
--shell=/bin/bash $DJANGO_USER
Here is why we use these parameters:
--system
- This tells
adduser
to create a system user, as opposed to creating a normal user. System users are intended to run programs, whereas normal users are people. Because of this parameter,adduser
will assign a user id less than 1000, which is only a convention for knowing that this is a system user. Otherwise there isn’t much difference. --home=/var/opt/$DJANGO_PROJECT
- This specifies the home directory for the user. For system users, it
doesn’t really matter which directory we will choose, but by
convention we choose the one which holds the program’s data. We will
talk about the
/var/opt/$DJANGO_PROJECT
directory later. --no-create-home
- We tell
adduser
to not create the home directory. We could allow it to create it, but we will create it ourselves later on, for instructive purposes. --disabled-password
- The password will be, well, disabled. This means that you won’t be
able to become this user by using a password. However, the root user
can always become another user (e.g. with
su
) without using a password, so we don’t need one. --group
- This tells
adduser
to not only add a new user, but to also add a new group, having the same name as the user, and make the new user a member of the new group. We will see further below why this is useful. I will be using$DJANGO_GROUP
to denote the new group. In principle it could be different than$DJANGO_USER
(but then the procedure of creating the user and the group would be slightly different), but the most important thing is that I want it to be perfectly clear when we are talking about the user and when we are talking about the group. --shell=/bin/bash
- By default,
adduser
uses/bin/false
as the shell for system users, which practically means they are disabled;/bin/false
can’t run any commands. We want the user to have the most common shell used in GNU/Linux systems,/bin/bash
.
3.2. The program files¶
Your Django project should be structured either like this:
$DJANGO_PROJECT/
|-- manage.py
|-- requirements.txt
|-- your_django_app/
`-- $DJANGO_PROJECT/
or like this:
$REPOSITORY_ROOT/
|-- requirements.txt
`-- $DJANGO_PROJECT/
|-- manage.py
|-- your_django_app/
`-- $DJANGO_PROJECT/
I prefer the former, but some people prefer the extra repository root directory.
We are going to place your project inside /opt
. This is a standard
directory for program files that are not part of the operating system.
(The ones that are installed by the operating system go to /usr
.)
So, clone or otherwise copy your Django project in
/opt/$DJANGO_PROJECT
or in /opt/$REPOSITORY_ROOT
. Do
this as the root user. Create the virtualenv for your project as
the root user as well:
virtualenv --system-site-packages --python=/usr/bin/python3 \
/opt/$DJANGO_PROJECT/venv
/opt/$DJANGO_PROJECT/venv/bin/pip install \
-r /opt/$DJANGO_PROJECT/requirements.txt
While it might seem strange that we are creating these as the root user
instead of as $DJANGO_USER
, it is standard practice
for program files to belong to the root user. If you check, you will see
that /bin/ls
belongs to the root user, though you may be running it
as joe. In fact, it would be an error for it to belong to joe, because
then joe would be able to modify it. So for security purposes it’s
better for program files to belong to root.
This poses a problem: when $DJANGO_USER
attempts to execute your
Django application, it will not have permission to write
the compiled Python files in the /opt/$DJANGO_PROJECT
directory,
because this is owned by root. So we need to pre-compile
these files as root:
/opt/$DJANGO_PROJECT/venv/bin/python -m compileall \
-x /opt/$DJANGO_PROJECT/venv/ /opt/$DJANGO_PROJECT
The option -x /opt/$DJANGO_PROJECT/venv/
tells compileall to exclude
directory /opt/$DJANGO_PROJECT/venv
from compilation. This is
because the virtualenv takes care of its own compilation and we should
not interfere.
3.3. The data directory¶
As I already hinted, our data directory is going to be
/var/opt/$DJANGO_PROJECT
. It is standard policy for programs
installed in /opt
to put their data in /var/opt
. Most notably,
we will store media files in there (in a later chapter). We will also
store the SQLite file there. Usually in production we use a
different RDBMS, but we will deal with this in a later chapter as well.
So, let’s now prepare the data directory:
mkdir -p /var/opt/$DJANGO_PROJECT
chown $DJANGO_USER /var/opt/$DJANGO_PROJECT
Besides creating the directory, we also changed its owner to
$DJANGO_USER
. This is necessary because Django will be needing to
write data in that directory, and it will be running as that user, so it
needs permission to do so.
3.4. The log directory¶
Later we will setup our Django project to write to log files in
/var/log/$DJANGO_PROJECT
. Let’s prepare the directory.
mkdir -p /var/log/$DJANGO_PROJECT
chown $DJANGO_USER /var/log/$DJANGO_PROJECT
3.5. The production settings¶
Debian puts configuration files in /etc
. More specifically, the
configuration for programs that are installed in /opt
is supposed to
go to /etc/opt
, which is what we will do:
mkdir /etc/opt/$DJANGO_PROJECT
For the time being this directory is going to have only settings.py
;
later it will have a bit more. Your
/etc/opt/$DJANGO_PROJECT/settings.py
file should be like this:
from DJANGO_PROJECT.settings import *
DEBUG = True
ALLOWED_HOSTS = ['$DOMAIN', 'www.$DOMAIN']
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/var/opt/$DJANGO_PROJECT/$DJANGO_PROJECT.db',
}
}
Note
The above is not valid Python until you replace $DJANGO_PROJECT
with the name of your django project and $DOMAIN
with your
domain. In all examples until now you might have been able to copy
and paste the code from the book and use shell variables for
$DJANGO_PROJECT
, $DJANGO_USER
, $DJANGO_GROUP
, and so on.
This is, indeed, the reason I chose this notation. However, in some
places, like in this Python, you have to actually replace it
yourself. (Occasionally I use DJANGO_PROJECT without the leading
dollar sign, in order to get the syntax highlighter to work.)
Note
These settings might give you the error “The SECRET_KEY setting must not be empty”, or “Unknown command: ‘collectstatic’”, or some other error that indicates a problem with the settings. If this happens, a likely explanation is that this line at the top of your production settings isn’t working correctly:
from DJANGO_PROJECT.settings import *
It may be that, in your Django project, settings
is a directory
that has no __init__.py
file or an empty __init__.py
file.
Maybe you have to change the line to this:
from DJANGO_PROJECT.settings.base import *
Check what your project’s settings file actually is, and import from that one.
Let’s now secure the production settings. We don’t want other users of the system to be able to read the file, because it contains sensitive information. Maybe not yet, but after a few chapters it is going to have the secret key, the password to the database, the password for the email server, etc. At this point, you are wondering: what other users? I am the only person using this server, and I have created no users. Indeed, now that it’s so easy and cheap to get small servers and assign a single job to them, this detail is not as important as it used to be. However, it is still a good idea to harden things a little bit. Maybe a year later you will create a normal user account on that server as an unrelated convenience for a colleague.
If your Django project has a vulnerability, an attacker might be able to
give commands to the system as the user as which the project runs (i.e.
as $DJANGO_USER
). Likewise, in the future you might install some
other web application, and that other web application might have a
vulnerability and could be attacked, and the attacker might be able to
give commands as the user running that application. In that case, if we
have secured our settings.py
, the attacker won’t be able to read it.
Eventually servers get compromised, and we try to set up the system in
such a way as to minimize the damage, and we can minimize it if we
contain it, and we can contain it if the compromising of an application
does not result in the compromising of other applications. This is why
we want to run each application in its own user and its own group.
Here is how to make the contents of /etc/opt/$DJANGO_PROJECT
unreadable by other users:
chgrp $DJANGO_GROUP /etc/opt/$DJANGO_PROJECT
chmod u=rwx,g=rx,o= /etc/opt/$DJANGO_PROJECT
What this does is make the directory unreadable by users other than
root
and $DJANGO_USER
. The directory is owned by root
, and
the first command above changes the group of the directory to
$DJANGO_GROUP
. The second command changes the permissions of the
directory so that:
- u=rwx
- The owner has permission to read (rx) and write (w) the directory
(the
u
inu=rwx
stands for “user”, but actually it means the “user who owns the directory”). The owner isroot
. Reading a directory is denoted withrx
rather than simplyr
, where thex
stands for “search”; but giving a directory only one of ther
andx
permissions is an edge case that I’ve seen only once in my life. For practical purposes, when you want a directory to be readable, you must specify bothr
andx
. (This applies only to directories; for files, thex
is the permission to execute the file as a program.) - g=rx
- The group has permission to read the directory. More precisely, users
who belong in that group have permission to read the directory. The
directory’s group is
$DJANGO_GROUP
. The only user in that group is$DJANGO_USER
, so this adjustment applies only to that user. - o=
- Other users have no permission, they can’t read or write to the directory.
You might have expected that it would have been easier to tell the
system “I want root
to be able to read and write, and
$DJANGO_USER
to be able to only read”. Instead, we did something
much more complicated: we made $DJANGO_USER
belong to a
$DJANGO_GROUP
, and we made the directory readable by that group,
thus indirectly readable by the user. The reason we did it this way is
an accident of history. In Unix there has traditionally been no way to
say “I want root
to be able to read and write, and $DJANGO_USER
to be able to only read”. In many modern Unixes, including Linux, it is
possible using Access Control Lists, but this is a feature added later,
it does not work the same in all Unixes, and its syntax is harder to
use. The way we use here works the same in FreeBSD, HP-UX, and all other
Unixes, and it is common practice everywhere.
Finally, we need to compile the settings file. Your settings file
and the /etc/opt/$DJANGO_PROJECT
directory is owned by root, and, as
with the files in /opt
, Django won’t be able to write the
compiled version, so we pre-compile it as root:
/opt/$DJANGO_PROJECT/venv/bin/python -m compileall \
/etc/opt/$DJANGO_PROJECT
Compiled files are the reason we changed the permissions of the
directory and not the permissions of settings.py
. When Python writes
the compiled files (which also contain the sensitive information), it
does not give them the permissions we want, which means we’d need to be
chgrping and chmoding each time we compile. By removing read permissions
from the directory, we make sure that none of the files in the directory
is readable; in Unix, in order to read file
/etc/opt/$DJANGO_PROJECT/settings.py
, you must have permission to
read /
(the root directory), /etc
, /etc/opt
,
/etc/opt/$DJANGO_PROJECT
, and
/etc/opt/$DJANGO_PROJECT/settings.py
.
You can check the permissions of a directory with the -d
option of
ls
, like this:
ls -lhd /
ls -lhd /etc
ls -lhd /etc/opt
ls -lhd /etc/opt/$DJANGO_PROJECT
(In the above commands, if you don’t use the -d
option it will show
the contents of the directory instead of the directory itself.)
Hint
Unix permissions
When you list a file or directory with the -l
option of ls
,
it will show you something like -rwxr-xr-x
at the beginning of
the line. The first character is the file type: -
for a file and
d
for a directory (there are also some more types, but we won’t
bother with them). The next nine characters are the permissions:
three for the user, three for the group, three for others.
rwxr-xr-x
means “the user has permission to read, write and
search/execute, the group has permission to read and search/execute
but not write, and so do others”.
rwxr-xr-x
can also be denoted as 755. If you substitute 0 in
place of a hyphen and 1 in place of r, w and x, you get 111 101 101.
In octal, this is 755. Instead of
chmod u=rwx,g=rx,o= /etc/opt/$DJANGO_PROJECT
you can type
chmod 750 /etc/opt/$DJANGO_PROJECT
which means exactly the same thing. People use this latter version much more than the other one, because it is so much easier to type, and because converting permissions into octal becomes second nature with a little practice.
3.6. Managing production vs. development settings¶
How to manage production vs. development settings seems to be an eternal
question. Many people recommend, instead of a single settings.py
file, a settings
directory containing __init__.py
and
base.py
. base.py
is the base settings, those that are the same
whether in production or development or testing. The directory often
contains local.py
(alternatively named dev.py
), with common
development settings, which might or might not be in the repository.
There’s often also test.py
, settings that are used when testing.
Both local.py
and test.py
start with this line:
from .base import *
Then they go on to override the base settings or add more settings.
When the project is set up like this, manage.py
is usually modified
so that, by default, it uses $DJANGO_PROJECT.settings.local
instead
of simply $DJANGO_PROJECT.settings
. For more information on this
technique, see Section 5.2, “Using Multiple Settings Files”, in the book
Two Scoops of Django; there’s also a stackoverflow answer about it.
Now, people who use this scheme sometimes also have production.py
in
the settings directory of the repository. Call me a perfectionist (with
deadlines), but the production settings are the administrator’s job, not
the developer’s, and your django project’s repository is made by the
developers. You might claim that you are both the developer and the
administrator, since it’s you who are developing the project and
maintaining the deployment, but in this case you are assuming two roles,
wearing a different hat each time. Production settings don’t belong in
the project repository any more than the nginx or PostgreSQL
configuration does.
The proper place to store such settings is another repository—the
deployment repository. It can be as simple as holding only the
production settings.py
(along with README
and .gitignore
),
or as complicated as containing all your nginx, PostgreSQL, etc.,
configuration for several servers, along with the “recipe” for how to
set them up, written with a configuration management system such as
Ansible.
If you choose, however, to keep your production settings in your Django
project repository, then your /etc/opt/$DJANGO_PROJECT/settings.py
file shall eventually be a single line:
from $DJANGO_PROJECT.settings.production import *
However, I don’t want you to do this now. We aren’t yet going to use our
real production settings, because we are going step by step. Instead,
create the /etc/opt/$DJANGO_PROJECT/settings.py
file as I explained
in the previous section.
3.7. Running the Django server¶
Warning
We are running Django with runserver
here, which is inappropriate
for production. We are doing it only temporarily, so that you
understand several concepts. We will run Django correctly in the
chapter about Gunicorn.
su $DJANGO_USER
source /opt/$DJANGO_PROJECT/venv/bin/activate
export PYTHONPATH=/etc/opt/$DJANGO_PROJECT:/opt/$DJANGO_PROJECT
export DJANGO_SETTINGS_MODULE=settings
python /opt/$DJANGO_PROJECT/manage.py migrate
python /opt/$DJANGO_PROJECT/manage.py runserver 0.0.0.0:8000
You could also do that in an exceptionally long command (provided you
have already done the migrate
part), like this:
PYTHONPATH=/etc/opt/$DJANGO_PROJECT:/opt/$DJANGO_PROJECT \
DJANGO_SETTINGS_MODULE=settings \
su $DJANGO_USER -c \
"/opt/$DJANGO_PROJECT/venv/bin/python \
/opt/$DJANGO_PROJECT/manage.py runserver 0.0.0.0:8000"
Hint
su
You have probably heard of sudo
, which is a very useful program
on Unix client machines (desktops and laptops). On the server,
sudo
is less common and we use su
instead.
su
, like sudo
, changes the user that executes a program. If
you are user joe and you execute su -c ls
, then ls
is run as
root. su
will ask for the root password in order to proceed.
su alice -c ls
means “execute ls
as user alice”. su alice
means “start a shell as user alice”; you can then type commands as
user alice, and you can enter exit
to “get out” of su
, that
is, to exit the shell than runs as alice. If you are a normal user
su
will ask you for alice’s password. If you are root, it will
become alice without questions. This should make clear how the su
command works when you run the Django server as explained above.
sudo
works very differently from su
. Instead of asking the
password of the user you want to become, it asks for your password,
and has a configuration file that describes which user is allowed to
become what user and with what constraints. It is much more
versatile. su
does only what I described and nothing more. su
is guaranteed to exist in all Unix systems, whereas sudo
is an
add-on that must be installed. By default it is usually installed on
client machines, but not on servers. su
is much more commonly
used on servers and shell scripts than sudo
.
Do you understand that very clearly? If not, here are some tips:
- Make sure you have a grip on virtualenv and environment variables.
- Python reads the
PYTHONPATH
environment variable and adds the specified directories to the Python path. - Django reads the
DJANGO_SETTINGS_MODULE
environment variable. Because we have set it to “settings”, Django will attempt to importsettings
instead of the default (the default is$DJANGO_PROJECT.settings
, or maybe$DJANGO_PROJECT.settings.local
). - When Django attempts to import
settings
, Python looks in its path. Because/etc/opt/$DJANGO_PROJECT
is listed first inPYTHONPATH
, Python will first look there forsettings.py
, and it will find it there. - Likewise, when at some point Django attempts to import
your_django_app
, Python will look in/etc/opt/$DJANGO_PROJECT
; it won’t find it there, so then it will look in/opt/$DJANGO_PROJECT
, since this is next inPYTHONPATH
, and it will find it there. - If, before running
manage.py [whatever]
, we had changed directory to/opt/$DJANGO_PROJECT
, we wouldn’t need to specify that directory inPYTHONPATH
, because Python always adds the current directory to its path. This is why, in development, you just tell itpython manage.py [whatever]
and it finds your project. We prefer, however, to set thePYTHONPATH
and not change directory; this way our setup will be clearer and more robust.
Instead of using DJANGO_SETTINGS_MODULE
, you can also use the
--settings
parameter of manage.py
:
PYTHONPATH=/etc/opt/$DJANGO_PROJECT:/opt/$DJANGO_PROJECT \
su $DJANGO_USER -c \
"/opt/$DJANGO_PROJECT/venv/bin/python \
/opt/$DJANGO_PROJECT/manage.py
runserver --settings=settings 0.0.0.0:8000"
(manage.py
also supports a --pythonpath
parameter which could be
used instead of PYTHONPATH
, however it seems that --settings
doesn’t work correctly together with --pythonpath
, at least not in
Django 1.8.)
If you fire up your browser and visit http://$DOMAIN:8000/, you should see your Django project in action.
3.8. Chapter summary¶
- Create a system user and group with the same name as your Django project.
- Put your Django project in
/opt
, with all files owned by root. - Put your virtualenv in
/opt/$DJANGO_PROJECT/venv
, with all files owned by root. - Put your data files in a subdirectory of
/var/opt
with the same name as your Django project, owned by the system user you created. If you are using SQLite, the database file will go in there. - Put your settings file in a subdirectory of
/etc/opt
with the same name as your Django project, whose user is root, whose group is the system group you created, that is readable by the group and writeable by root, and whose contents belong to root. - Precompile the files in
/opt/$DJANGO_PROJECT
and/etc/opt/$DJANGO_PROJECT
. - Run
manage.py
as the system user you created, after setting the environment variablesPYTHONPATH=/etc/opt/$DJANGO_PROJECT:/opt/$DJANGO_PROJECT
andDJANGO_SETTINGS_MODULE=settings
.