diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..53a4e81 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +*.sql.gz diff --git a/README.md b/README.md index c141d6d..ebdb7c5 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,16 @@ # netbox-docker -[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)](https://travis-ci.org/ninech/netbox-docker) +[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)][travis] -This repository houses the components needed to build NetBox as a Docker container. -Images built using this code are released to [Docker Hub](https://hub.docker.com/r/ninech/netbox) every night. +This repository houses the components needed to build Netbox as a Docker container. +Images built using this code are released to [Docker Hub][netbox-dockerhub] every night. + +[travis]: https://travis-ci.org/ninech/netbox-docker +[netbox-dockerhub]: https://hub.docker.com/r/ninech/netbox/tags/ ## Quickstart -To get NetBox up and running: +To get Netbox up and running: ``` $ git clone -b master https://github.com/ninech/netbox-docker.git @@ -30,7 +33,7 @@ $ open "http://$(docker-compose port nginx 80)/" $ xdg-open "http://$(docker-compose port nginx 80)/" &>/dev/null & ``` -Alternatively, use something like [Reception](https://github.com/ninech/reception) to +Alternatively, use something like [Reception][docker-reception] to connect to _docker-compose_ projects. Default credentials: @@ -39,6 +42,8 @@ Default credentials: * Password: **admin** * API Token: **0123456789abcdef0123456789abcdef01234567** +[docker-reception]: https://github.com/ninech/reception + ## Dependencies This project relies only on *Docker* and *docker-compose* meeting this requirements: @@ -51,13 +56,74 @@ To ensure this, compare the output of `docker --version` and `docker-compose --v ## Configuration You can configure the app using environment variables. These are defined in `netbox.env`. +Read [Environment Variables in Compose][compose-env] to understand about the various possibilities to overwrite these variables. +(The easiest solution being simply adjusting that file.) + +To find all possible variables, have a look at the [configuration.docker.py][docker-config] and [docker-entrypoint.sh][entrypoint] files. +Generally, the environment variables are called the same as their respective Netbox configuration variables. +Variables which are arrays are usually composed by putting all the values into the same environment variables with the values separated by a whitespace ("` `"). +For example defining `ALLOWED_HOSTS=localhost ::1 127.0.0.1` would allows access to Netbox through `http://localhost`, `http://[::1]` and `http://127.0.0.1`. + +[compose-env]: https://docs.docker.com/compose/environment-variables/ + +### Production + +The default settings are optimized for (local) development environments. +You should therefore adjust the configuration for production setups, at least the following variables: + +* `ALLOWED_HOSTS`: Add all URLs that lead to your netbox instance. +* `DB_*`: Use a persistent database. +* `EMAIL_*`: Use your own mailserver. +* `MAX_PAGE_SIZE`: Use the recommended default of 1000. +* `SUPERUSER_*`: Only define those variables during the initial setup, and drop them once the DB is set up. + +### Running on Docker Swarm / Kubernetes / OpenShift + +You may run this image in a cluster such as Docker Swarm, Kubernetes or OpenShift, but this is advanced level. + +In this case, we encourage you to statically configure Netbox by starting from [Netbox's example config file][default-config], and mounting it into your container using the mechanism provided by your container platform (i.e. [Docker Swarm configs][swarm-config], [Kubernetes secrets][k8s-secrets], [OpenShift configmaps][openshift-config]). + +But if you rather continue to configure your application through environment variables, you may continue to use [the built-in configuration file][docker-config]. +We discourage storing secrets in environment variables, as environment variable are passed on to all sub-processes and may leak easily into other systems, e.g. error collecting tools that often collect all environment variables whenever an error occurs. + +Therefore we *strongly advise* to make use of the secrets mechanism provided by your container platform (i.e. [Docker Swarm secrets][swarm-secrets], [Kubernetes secrets][k8s-secrets], [OpenShift secrets][openshift-secrets]). +[The configuration file][docker-config] and [the entrypoint script][entrypoint] try to load the following secrets from the respective files. +If a secret is defined by an environment variable and in the respective file at the same time, then the value from the environment variable is used. + +* `SUPERUSER_PASSWORD`: `/run/secrets/superuser_password` +* `SUPERUSER_API_TOKEN`: `/run/secrets/superuser_api_token` +* `DB_PASSWORD`: `/run/secrets/db_password` +* `SECRET_KEY`: `/run/secrets/secret_key` +* `EMAIL_PASSWORD`: `/run/secrets/email_password` +* `NAPALM_PASSWORD`: `/run/secrets/napalm_password` + +Please also consider [the advice about running Netbox in production](#production) above! + +[docker-config]: https://github.com/ninech/netbox-docker/blob/master/docker/configuration.docker.py +[default-config]: https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py +[entrypoint]: https://github.com/ninech/netbox-docker/blob/master/docker/docker-entrypoint.sh +[swarm-config]: https://docs.docker.com/engine/swarm/configs/ +[swarm-secrets]: https://docs.docker.com/engine/swarm/secrets/ +[openshift-config]: https://docs.openshift.org/latest/dev_guide/configmaps.html +[openshift-secrets]: https://docs.openshift.org/latest/dev_guide/secrets.html +[k8s-secrets]: https://kubernetes.io/docs/concepts/configuration/secret/ + +#### A Note On OpenShift + +OpenShift usually is configured with specific restriction regarding root users. +[Special care][openshift-root] has to be taken when building images for OpenShift. +The Docker Image that may be built using this project (and which is available on Docker Hub) might not yet run without further customization on OpenShift. +If you have this running on OpenShift, it would be nice if you could open a PR with the changes you needed to make. +Or if you didn't do any changes and it just worked, that you could confirm this so that we can remove this notice. + +[openshift-root]: https://docs.openshift.org/latest/creating_images/guidelines.html#openshift-specific-guidelines ## Version The `docker-compose.yml` file is prepared to run a specific version of Netbox. To use this feature, set the environment-variable `VERSION` before launching `docker-compose`, as shown below. `VERSION` may be set to the name of -[any tag of the `ninech/netbox` Docker image](https://hub.docker.com/r/ninech/netbox/tags/). +[any tag of the `ninech/netbox` Docker image on Docker Hub][netbox-dockerhub]. ``` $ export VERSION=v2.2.6 @@ -66,9 +132,8 @@ $ docker-compose up -d ``` You can also build a specific version of the Netbox image. This time, `VERSION` indicates any valid -[Git Reference](https://git-scm.com/book/en/v2/Git-Internals-Git-References) declared on -[the Netbox Github repository](https://github.com/digitalocean/netbox/releases). -Most commonly you will specify a tag name or a branch name. +[Git Reference][git-ref] declared on [the 'digitalocean/netbox' Github repository][netbox-github]. +Most commonly you will specify a tag or branch name. ``` $ export VERSION=develop @@ -79,6 +144,47 @@ $ docker-compose up -d Hint: If you're building a specific version by tag name, the `--no-cache` argument is not strictly necessary. This can increase the build speed if you're just adjusting the config, for example. +[git-ref]: https://git-scm.com/book/en/v2/Git-Internals-Git-References +[netbox-github]: https://github.com/digitalocean/netbox/releases + +## Troubleshooting + +This section is a collection of some common issues and how to resolve them. +If your issue is not here, look through [the existing issues][issues] and eventually create a new issue. + +[issues]: (https://github.com/ninech/netbox-docker/issues) + +### Docker Compose basics + +* You can see all running containers belonging to this project using `docker-compose ps`. +* You can see the logs by running `docker-compose logs -f`. + Running `docker-compose logs -f netbox` will just show the logs for netbox. +* You can stop everything using `docker-compose stop`. +* You can clean up everything using `docker-compose down -v --remove-orphans`. **This will also remove any related data.** +* You can enter the shell of the running Netbox container using `docker-compose exec netbox /bin/bash`. Now you have access to `./manage.py`, e.g. to reset a password. +* To access the database run `docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'` +* To create a database backup run `docker-compose exec postgres sh -c 'pg_dump -U $POSTGRES_USER $POSTGRES_DB' | gzip > db_dump.sql.gz` +* To restore that database backup run `gunzip -c db_dump.sql.gz | docker exec -i $(docker-compose ps -q postgres) sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'`. + +### Getting a "Bad Request (400)" + +> When connecting to the Netbox instance, I get a "Bad Request (400)" error. + +This usually happens when the `ALLOWED_HOSTS` variable is not set correctly. + +### How to upgrade + +> How do I update to a newer version? + +It should be sufficient to pull the latest image from Docker Hub, stopping the container and starting it up again: + +```bash +docker-compose pull netbox +docker-compose stop netbox +docker-compose rm -f netbox +docker-compose up -d netbox +``` + ## Rebuilding & Publishing images `./build.sh` is used to rebuild the Docker image: @@ -104,7 +210,7 @@ You can use the following ENV variables to customize the build: ## Tests -To run the bundled test, use the `docker-compose.test.yml` file. +To run the test coming with Netbox, use the `docker-compose.test.yml` file as such: ``` $ docker-compose -f docker-compose.test.yml run --rm app diff --git a/docker-compose.test.yml b/docker-compose.test.yml index 978de00..9177dd2 100644 --- a/docker-compose.test.yml +++ b/docker-compose.test.yml @@ -1,23 +1,22 @@ version: '3' services: app: - build: - context: . - args: - - BRANCH=${BRANCH-master} - image: ninech/netbox:${BRANCH-latest} - depends_on: - - postgres - env_file: netbox.env - command: - - ./manage.py - - test + build: + context: . + args: + - BRANCH=${BRANCH-master} + image: ninech/netbox:${BRANCH-latest} + depends_on: + - postgres + env_file: netbox.env + command: + - ./manage.py + - test postgres: - image: postgres:9.6-alpine - env_file: postgres.env - + image: postgres:9.6-alpine + env_file: postgres.env volumes: - netbox-static-files: - driver: local - netbox-nginx-config: - driver: local + netbox-static-files: + driver: local + netbox-nginx-config: + driver: local diff --git a/docker-compose.yml b/docker-compose.yml index d34b591..f76a7b3 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -12,13 +12,15 @@ services: volumes: - netbox-nginx-config:/etc/netbox-nginx/ - netbox-static-files:/opt/netbox/netbox/static + - netbox-media-files:/opt/netbox/netbox/media + - netbox-report-files:/opt/netbox/netbox/reports nginx: image: nginx:1.11-alpine command: nginx -g 'daemon off;' -c /etc/netbox-nginx/nginx.conf depends_on: - netbox ports: - - 80 + - 8080 volumes: - netbox-static-files:/opt/netbox/netbox/static - netbox-nginx-config:/etc/netbox-nginx/ @@ -31,3 +33,7 @@ volumes: driver: local netbox-nginx-config: driver: local + netbox-media-files: + driver: local + netbox-report-files: + driver: local diff --git a/docker/configuration.docker.py b/docker/configuration.docker.py index 7ad16f4..f973809 100644 --- a/docker/configuration.docker.py +++ b/docker/configuration.docker.py @@ -1,4 +1,21 @@ import os +import socket + +# For reference see http://netbox.readthedocs.io/en/latest/configuration/mandatory-settings/ +# Based on https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py + +# Read secret from file +def read_secret(secret_name): + try: + f = open('/run/secrets/' + secret_name, 'r', encoding='utf-8') + except EnvironmentError: + return '' + else: + with f: + return f.readline().strip() + +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + ######################### # # # Required settings # @@ -9,13 +26,14 @@ import os # access to the server via any other hostnames. The first FQDN in the list will be treated as the preferred name. # # Example: ALLOWED_HOSTS = ['netbox.example.com', 'netbox.internal.local'] -ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(' ') +ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', socket.gethostname()).split(' ') # PostgreSQL database configuration. DATABASE = { 'NAME': os.environ.get('DB_NAME', 'netbox'), # Database name 'USER': os.environ.get('DB_USER', ''), # PostgreSQL username - 'PASSWORD': os.environ.get('DB_PASSWORD', ''), # PostgreSQL password + 'PASSWORD': os.environ.get('DB_PASSWORD', read_secret('db_password')), + # PostgreSQL password 'HOST': os.environ.get('DB_HOST', 'localhost'), # Database server 'PORT': os.environ.get('DB_PORT', ''), # Database port (leave blank for default) } @@ -24,7 +42,7 @@ DATABASE = { # For optimal security, SECRET_KEY should be at least 50 characters in length and contain a mix of letters, numbers, and # symbols. NetBox will not run without this defined. For more information, see # https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECRET_KEY -SECRET_KEY = os.environ.get('SECRET_KEY', '') +SECRET_KEY = os.environ.get('SECRET_KEY', read_secret('secret_key')) ######################### # # @@ -38,16 +56,51 @@ ADMINS = [ # ['John Doe', 'jdoe@example.com'], ] +# Optionally display a persistent banner at the top and/or bottom of every page. HTML is allowed. To display the same +# content in both banners, define BANNER_TOP and set BANNER_BOTTOM = BANNER_TOP. +BANNER_TOP = os.environ.get('BANNER_TOP', '') +BANNER_BOTTOM = os.environ.get('BANNER_BOTTOM', '') + +# Text to include on the login page above the login form. HTML is allowed. +BANNER_LOGIN = os.environ.get('BANNER_LOGIN', '') + +# Base URL path if accessing NetBox within a directory. For example, if installed at http://example.com/netbox/, set: +# BASE_PATH = 'netbox/' +BASE_PATH = os.environ.get('BASE_PATH', '') + +# API Cross-Origin Resource Sharing (CORS) settings. If CORS_ORIGIN_ALLOW_ALL is set to True, all origins will be +# allowed. Otherwise, define a list of allowed origins using either CORS_ORIGIN_WHITELIST or +# CORS_ORIGIN_REGEX_WHITELIST. For more information, see https://github.com/ottoyiu/django-cors-headers +CORS_ORIGIN_ALLOW_ALL = os.environ.get('CORS_ORIGIN_ALLOW_ALL', False) +CORS_ORIGIN_WHITELIST = os.environ.get('CORS_ORIGIN_WHITELIST', '').split(' ') +CORS_ORIGIN_REGEX_WHITELIST = [ + # r'^(https?://)?(\w+\.)?example\.com$', +] + +# Set to True to enable server debugging. WARNING: Debugging introduces a substantial performance penalty and may reveal +# sensitive information about your installation. Only enable debugging while performing testing. Never enable debugging +# on a production system. +DEBUG = os.environ.get('DEBUG', False) + # Email settings EMAIL = { 'SERVER': os.environ.get('EMAIL_SERVER', 'localhost'), 'PORT': os.environ.get('EMAIL_PORT', 25), 'USERNAME': os.environ.get('EMAIL_USERNAME', ''), - 'PASSWORD': os.environ.get('EMAIL_PASSWORD', ''), + 'PASSWORD': os.environ.get('EMAIL_PASSWORD', read_secret('email_password')), 'TIMEOUT': os.environ.get('EMAIL_TIMEOUT', 10), # seconds 'FROM_EMAIL': os.environ.get('EMAIL_FROM', ''), } +# Enforcement of unique IP space can be toggled on a per-VRF basis. +# To enforce unique IP space within the global table (all prefixes and IP addresses not assigned to a VRF), +# set ENFORCE_GLOBAL_UNIQUE to True. +ENFORCE_GLOBAL_UNIQUE = os.environ.get('ENFORCE_GLOBAL_UNIQUE', False) + +# Enable custom logging. Please see the Django documentation for detailed guidance on configuring custom logs: +# https://docs.djangoproject.com/en/1.11/topics/logging/ +LOGGING = {} + # Setting this to True will permit only authenticated users to access any part of NetBox. By default, anonymous users # are permitted to access most data in NetBox (excluding secrets) but not make any changes. LOGIN_REQUIRED = os.environ.get('LOGIN_REQUIRED', False) @@ -59,13 +112,37 @@ BASE_PATH = os.environ.get('BASE_PATH', '') # Setting this to True will display a "maintenance mode" banner at the top of every page. MAINTENANCE_MODE = os.environ.get('MAINTENANCE_MODE', False) +# An API consumer can request an arbitrary number of objects =by appending the "limit" parameter to the URL (e.g. +# "?limit=1000"). This setting defines the maximum limit. Setting it to 0 or None will allow an API consumer to request +# all objects by specifying "?limit=0". +MAX_PAGE_SIZE = int(os.environ.get('MAX_PAGE_SIZE', 1000)) + +# The file path where uploaded media such as image attachments are stored. A trailing slash is not needed. Note that +# the default value of this setting is derived from the installed location. +MEDIA_ROOT = os.environ.get('MEDIA_ROOT', os.path.join(BASE_DIR, 'media')) + # Credentials that NetBox will use to access live devices. NAPALM_USERNAME = os.environ.get('NAPALM_USERNAME', '') -NAPALM_PASSWORD = os.environ.get('NAPALM_PASSWORD', '') +NAPALM_PASSWORD = os.environ.get('NAPALM_PASSWORD', read_secret('napalm_password')) + +# NAPALM timeout (in seconds). (Default: 30) +NAPALM_TIMEOUT = os.environ.get('NAPALM_TIMEOUT', 30) + +# NAPALM optional arguments (see http://napalm.readthedocs.io/en/latest/support/#optional-arguments). Arguments must +# be provided as a dictionary. +NAPALM_ARGS = {} # Determine how many objects to display per page within a list. (Default: 50) PAGINATE_COUNT = os.environ.get('PAGINATE_COUNT', 50) +# When determining the primary IP address for a device, IPv6 is preferred over IPv4 by default. Set this to True to +# prefer IPv4 instead. +PREFER_IPV4 = os.environ.get('PREFER_IPV4', False) + +# The file path where custom reports will be stored. A trailing slash is not needed. Note that the default value of +# this setting is derived from the installed location. +REPORTS_ROOT = os.environ.get('REPORTS_ROOT', os.path.join(BASE_DIR, 'reports')) + # Time zone (default: UTC) TIME_ZONE = os.environ.get('TIME_ZONE', 'UTC') @@ -77,9 +154,3 @@ TIME_FORMAT = os.environ.get('TIME_FORMAT', 'g:i a') SHORT_TIME_FORMAT = os.environ.get('SHORT_TIME_FORMAT', 'H:i:s') DATETIME_FORMAT = os.environ.get('DATETIME_FORMAT', 'N j, Y g:i a') SHORT_DATETIME_FORMAT = os.environ.get('SHORT_DATETIME_FORMAT', 'Y-m-d H:i') - -# Enforcement of unique IP space can be toggled on a per-VRF basis. -# To enforce unique IP space within the global table (all prefixes and IP addresses not assigned to a VRF), -# set ENFORCE_GLOBAL_UNIQUE to True. -ENFORCE_GLOBAL_UNIQUE = os.environ.get('ENFORCE_GLOBAL_UNIQUE') - diff --git a/docker/docker-entrypoint.sh b/docker/docker-entrypoint.sh index f58b9bf..f03b760 100755 --- a/docker/docker-entrypoint.sh +++ b/docker/docker-entrypoint.sh @@ -8,17 +8,25 @@ while ! ./manage.py migrate 2>&1; do done # create superuser silently -if [[ -z ${SUPERUSER_NAME} ]]; then +if [ -z ${SUPERUSER_NAME+x} ]; then SUPERUSER_NAME='admin' fi -if [[ -z ${SUPERUSER_EMAIL} ]]; then +if [ -z ${SUPERUSER_EMAIL+x} ]; then SUPERUSER_EMAIL='admin@example.com' fi -if [[ -z ${SUPERUSER_PASSWORD} ]]; then - SUPERUSER_PASSWORD='admin' +if [ -z ${SUPERUSER_PASSWORD+x} ]; then + if [ -f "/run/secrets/superuser_password" ]; then + SUPERUSER_PASSWORD="$(< /run/secrets/superuser_password)" + else + SUPERUSER_PASSWORD='admin' + fi fi -if [[ -z ${SUPERUSER_API_TOKEN} ]]; then - SUPERUSER_API_TOKEN='0123456789abcdef0123456789abcdef01234567' +if [ -z ${SUPERUSER_API_TOKEN+x} ]; then + if [ -f "/run/secrets/superuser_api_token" ]; then + SUPERUSER_API_TOKEN="$(< /run/secrets/superuser_api_token)" + else + SUPERUSER_API_TOKEN='0123456789abcdef0123456789abcdef01234567' + fi fi echo "💡 Username: ${SUPERUSER_NAME}, E-Mail: ${SUPERUSER_EMAIL}, Password: ${SUPERUSER_PASSWORD}, Token: ${SUPERUSER_API_TOKEN}" diff --git a/docker/nginx.conf b/docker/nginx.conf index e736490..02efd01 100644 --- a/docker/nginx.conf +++ b/docker/nginx.conf @@ -1,25 +1,23 @@ worker_processes 1; events { - worker_connections 1024; + worker_connections 1024; } http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - sendfile on; - tcp_nopush on; - keepalive_timeout 65; - gzip on; - server_tokens off; + include /etc/nginx/mime.types; + default_type application/octet-stream; + sendfile on; + tcp_nopush on; + keepalive_timeout 65; + gzip on; + server_tokens off; client_max_body_size 10M; server { - listen 80; - + listen 8080; server_name localhost; - - access_log off; + access_log off; location /static/ { alias /opt/netbox/netbox/static/; diff --git a/netbox.env b/netbox.env index 9bed821..72bdb4a 100644 --- a/netbox.env +++ b/netbox.env @@ -1,18 +1,18 @@ -SUPERUSER_NAME=admin -SUPERUSER_EMAIL=admin@example.com -SUPERUSER_PASSWORD=admin -SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567 ALLOWED_HOSTS=localhost 0.0.0.0 127.0.0.1 [::1] netbox nginx netboxdocker.docker nginx.netboxdocker.docker DB_NAME=netbox DB_USER=netbox DB_PASSWORD=J5brHrAXFLQSif0K DB_HOST=postgres -SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj EMAIL_SERVER=localhost EMAIL_PORT=25 -EMAIL_USERNAME=foo -EMAIL_PASSWORD=bar -EMAIL_TIMEOUT=10 +EMAIL_USERNAME=netbox +EMAIL_PASSWORD= +EMAIL_TIMEOUT=5 EMAIL_FROM=netbox@bar.com -NETBOX_USERNAME=guest -NETBOX_PASSWORD=guest +NAPALM_TIMEOUT=5 +MAX_PAGE_SIZE=0 +SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj +SUPERUSER_NAME=admin +SUPERUSER_EMAIL=admin@example.com +SUPERUSER_PASSWORD=admin +SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567